Skip to contentSkip to conversation

EFI from usb HDD

DISCUSSION Michal Simek 4 participants 19 messages Jun 10, 2021 – Aug 19, 2021

lore
Share & tools
lore
#0 lore raw

Hi,

I am playing with booting from USB via EFI. And I see very weird behavior. I have burnt image with grub to USB flashdisk and I have tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board. On zcu102 grub is going to boot menu and everything is working fine as expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

On zcu102(working board) I also see usb(gpt) partitions and SD. grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

On zcu104 I see one more error message
"PE image measurement failed"
But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they are the same. SOM doesn't have usb enabled by default (but I enabled it) but grub starts which means that communication with USB is fine.

It is based on my latest patches available here. u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine. ZynqMP> part list usb 0

Partition Map for USB device 0 -- Partition Type: EFI

Part Start LBA End LBA Name
Attributes
Type GUID
Partition GUID
1 0x00000800 0x001007fe "Microsoft basic data"

        attrs:  0x0000000000000000
        type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
        type:   data
        guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77
  2     0x00100800      0x001197fe      "Microsoft basic data"
        attrs:  0x0000000000000000
        type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
        type:   data
        guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Do you have any idea why on one system is working fine to get to menu and on others there is an issue to get all partitions even u-boot is able to see them and can work with them.

Thanks,
Michal

#1 lore raw
Quoted 11 lines · Michal Simek

Hi,

I am playing with booting from USB via EFI. And I see very weird behavior. I have burnt image with grub to USB flashdisk and I have tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board. On zcu102 grub is going to boot menu and everything is working fine as expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

Quoted 5 lines

On zcu102(working board) I also see usb(gpt) partitions and SD. grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

Quoted 3 lines

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This will not stop disk enumeration.

Quoted 33 lines

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they are the same. SOM doesn't have usb enabled by default (but I enabled it) but grub starts which means that communication with USB is fine.

It is based on my latest patches available here. u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine. ZynqMP> part list usb 0

Partition Map for USB device 0 -- Partition Type: EFI

Part Start LBA End LBA Name
Attributes
Type GUID
Partition GUID
1 0x00000800 0x001007fe "Microsoft basic data"

         attrs:  0x0000000000000000
         type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
         type:   data
         guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77
   2     0x00100800      0x001197fe      "Microsoft basic data"
         attrs:  0x0000000000000000
         type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
         type:   data
         guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7
Quoted 7 lines

Do you have any idea why on one system is working fine to get to menu and on others there is an issue to get all partitions even u-boot is able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could be that the USB sub-system is simply not initialized yet when the boot manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough to run

efidebug devices

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Block devices are enumerated in efi_disk_register(). Please, try to add debug output there to elucidate the problem.

Best regards

Heinrich

#2 lore raw

Hi,

Quoted 83 lines · Heinrich Schuchardt
Quoted 11 lines · Michal Simek

Hi,

I am playing with booting from USB via EFI. And I see very weird behavior. I have burnt image with grub to USB flashdisk and I have tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board. On zcu102 grub is going to boot menu and everything is working fine as expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

Quoted 5 lines

On zcu102(working board) I also see usb(gpt) partitions and SD. grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

Quoted 3 lines

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This will not stop disk enumeration.

Quoted 33 lines

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they are the same. SOM doesn't have usb enabled by default (but I enabled it) but grub starts which means that communication with USB is fine.

It is based on my latest patches available here. u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine. ZynqMP> part list usb 0

Partition Map for USB device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name Attributes
         Type GUID
         Partition GUID
   1     0x00000800      0x001007fe      "Microsoft basic data" attrs:  0x0000000000000000
         type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
         guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77 2     0x00100800      0x001197fe      "Microsoft basic data" attrs:  0x0000000000000000
         type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
         guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Quoted 7 lines

Do you have any idea why on one system is working fine to get to menu and on others there is an issue to get all partitions even u-boot is able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could be that the USB sub-system is simply not initialized yet when the boot manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough to run

    efidebug devices

Quoted 8 lines

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Block devices are enumerated in efi_disk_register(). Please, try to add debug output there to elucidate the problem.

I found where the problem is. First of all zcu102 didn't use the same image as others (it wasn't updated properly).
When you have CONFIG_EFI_CAPSULE_ON_DISK_EARLY that efi_disk_register() is called before usb block devices are detected and registered that's why grub doesn't see them.
I was looking at adding usb start in preboot but preboot is called later. How this should be solved? Any idea?

Thanks,
Michal

#3 lore raw
Quoted 92 lines · Michal Simek

Hi,

Quoted 89 lines · Heinrich Schuchardt
Quoted 11 lines · Michal Simek

Hi,

I am playing with booting from USB via EFI. And I see very weird behavior. I have burnt image with grub to USB flashdisk and I have tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board. On zcu102 grub is going to boot menu and everything is working fine as expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

Quoted 5 lines

On zcu102(working board) I also see usb(gpt) partitions and SD. grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

Quoted 3 lines

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This will not stop disk enumeration.

Quoted 33 lines

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they are the same. SOM doesn't have usb enabled by default (but I enabled it) but grub starts which means that communication with USB is fine.

It is based on my latest patches available here. u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine. ZynqMP> part list usb 0

Partition Map for USB device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name
         Attributes
         Type GUID
         Partition GUID
   1     0x00000800      0x001007fe      "Microsoft basic data" attrs:  0x0000000000000000
         type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
         guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77 2     0x00100800      0x001197fe      "Microsoft basic data" attrs:  0x0000000000000000
         type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
         guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Quoted 7 lines

Do you have any idea why on one system is working fine to get to menu and on others there is an issue to get all partitions even u-boot is able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could be that the USB sub-system is simply not initialized yet when the boot manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough to run

    efidebug devices

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Quoted 8 lines
Quoted 2 lines

Block devices are enumerated in efi_disk_register(). Please, try to add debug output there to elucidate the problem.

I found where the problem is. First of all zcu102 didn't use the same image as others (it wasn't updated properly).
When you have CONFIG_EFI_CAPSULE_ON_DISK_EARLY that efi_disk_register() is called before usb block devices are detected and registered that's why grub doesn't see them.

The problem is CONFIG_EFI_SETUP_EARLY=y required by CONFIG_EFI_CAPSULE_ON_DISK_EARLY.

Why is USB initialized later then MMC?

Overall we have a deficiency in the UEFI implementation in that we cannot deal with block devices added or removed after initialization.

Here integration with the driver model is missing.

Quoted 7 lines

I was looking at adding usb start in preboot but preboot is called later. How this should be solved? Any idea?

Thanks,
Michal

+cc Sughosh, Takahiro (who have developed the capsule code)

Best regards

Heinrich

#4 lore raw
Quoted 106 lines · Heinrich Schuchardt
Quoted 96 lines · Michal Simek

Hi,

Quoted 93 lines · Heinrich Schuchardt
Quoted 11 lines · Michal Simek

Hi,

I am playing with booting from USB via EFI. And I see very weird behavior. I have burnt image with grub to USB flashdisk and I have tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board. On zcu102 grub is going to boot menu and everything is working fine as expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

Quoted 5 lines

On zcu102(working board) I also see usb(gpt) partitions and SD. grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

Quoted 3 lines

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This will not stop disk enumeration.

Quoted 35 lines

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they are
the same. SOM doesn't have usb enabled by default (but I enabled it) but
grub starts which means that communication with USB is fine.

It is based on my latest patches available here. u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine. ZynqMP> part list usb 0

Partition Map for USB device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name Attributes
          Type GUID
          Partition GUID
    1     0x00000800      0x001007fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77 2     0x00100800      0x001197fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Quoted 7 lines

Do you have any idea why on one system is working fine to get to menu and on others there is an issue to get all partitions even u-boot is able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could be that the USB sub-system is simply not initialized yet when the boot manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough to run

     efidebug devices

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Quoted 8 lines
Quoted 2 lines

Block devices are enumerated in efi_disk_register(). Please, try to add debug output there to elucidate the problem.

I found where the problem is. First of all zcu102 didn't use the same image as others (it wasn't updated properly).
When you have CONFIG_EFI_CAPSULE_ON_DISK_EARLY that efi_disk_register() is called before usb block devices are detected and registered that's why grub doesn't see them.

Quoted 4 lines

The problem is CONFIG_EFI_SETUP_EARLY=y required by CONFIG_EFI_CAPSULE_ON_DISK_EARLY.

Why is USB initialized later then MMC?

It is not just usb. SCSI/sata are behaving in the same way too.

Quoted 5 lines

Overall we have a deficiency in the UEFI implementation in that we cannot deal with block devices added or removed after initialization.

Here integration with the driver model is missing.

Right. And also there are commands which can create MBR partitions and I expect when you write image to SD and then run rescan or so you could get other partitions too.
Maybe hook via part_init()? with removing efi_disk_register.

Quoted 9 lines
Quoted 8 lines

I was looking at adding usb start in preboot but preboot is called later. How this should be solved? Any idea?

Thanks,
Michal

+cc Sughosh, Takahiro (who have developed the capsule code)

Thanks,
Michal

#5 lore raw
Quoted 120 lines · Michal Simek
Quoted 106 lines · Heinrich Schuchardt
Quoted 96 lines · Michal Simek

Hi,

Quoted 93 lines · Heinrich Schuchardt
Quoted 11 lines · Michal Simek

Hi,

I am playing with booting from USB via EFI. And I see very weird behavior. I have burnt image with grub to USB flashdisk and I have tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board. On zcu102 grub is going to boot menu and everything is working fine as expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

Quoted 5 lines

On zcu102(working board) I also see usb(gpt) partitions and SD. grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

Quoted 3 lines

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This will not stop disk enumeration.

Quoted 35 lines

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they are
the same. SOM doesn't have usb enabled by default (but I enabled it) but
grub starts which means that communication with USB is fine.

It is based on my latest patches available here. u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine. ZynqMP> part list usb 0

Partition Map for USB device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name Attributes
          Type GUID
          Partition GUID
    1     0x00000800      0x001007fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77 2     0x00100800      0x001197fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Quoted 7 lines

Do you have any idea why on one system is working fine to get to menu and on others there is an issue to get all partitions even u-boot is able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could be that the USB sub-system is simply not initialized yet when the boot manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough to run

     efidebug devices

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Quoted 8 lines
Quoted 2 lines

Block devices are enumerated in efi_disk_register(). Please, try to add debug output there to elucidate the problem.

I found where the problem is. First of all zcu102 didn't use the same image as others (it wasn't updated properly).
When you have CONFIG_EFI_CAPSULE_ON_DISK_EARLY that efi_disk_register() is called before usb block devices are detected and registered that's why grub doesn't see them.

Quoted 4 lines

The problem is CONFIG_EFI_SETUP_EARLY=y required by CONFIG_EFI_CAPSULE_ON_DISK_EARLY.

Why is USB initialized later then MMC?

It is not just usb. SCSI/sata are behaving in the same way too.

Quoted 4 lines

Overall we have a deficiency in the UEFI implementation in that we cannot deal with block devices added or removed after initialization.

Quoted 6 lines
Quoted 1 line

Here integration with the driver model is missing.

Right. And also there are commands which can create MBR partitions and I expect when you write image to SD and then run rescan or so you could get other partitions too.
Maybe hook via part_init()? with removing efi_disk_register.

For the record, I have proposed my ideas several times[1], [2]. I'm, however, no longer working on this issue as I have shifted my focus to UEFI secure boot and capsule update.

-Takahiro Akashi

[1] https://lists.denx.de/pipermail/u-boot/2018-November/347491.html
[2] https://lists.denx.de/pipermail/u-boot/2019-February/357923.html

Quoted 12 lines
Quoted 9 lines
Quoted 8 lines

I was looking at adding usb start in preboot but preboot is called later. How this should be solved? Any idea?

Thanks,
Michal

+cc Sughosh, Takahiro (who have developed the capsule code)

Thanks,
Michal

#6 lore raw

Hi,

Quoted 128 lines · AKASHI Takahiro
Quoted 120 lines · Michal Simek
Quoted 106 lines · Heinrich Schuchardt
Quoted 96 lines · Michal Simek

Hi,

Quoted 93 lines · Heinrich Schuchardt
Quoted 11 lines · Michal Simek

Hi,

I am playing with booting from USB via EFI. And I see very weird behavior. I have burnt image with grub to USB flashdisk and I have tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board. On zcu102 grub is going to boot menu and everything is working fine as expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

Quoted 5 lines

On zcu102(working board) I also see usb(gpt) partitions and SD. grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

Quoted 3 lines

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This will not stop disk enumeration.

Quoted 35 lines

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they are
the same. SOM doesn't have usb enabled by default (but I enabled it) but
grub starts which means that communication with USB is fine.

It is based on my latest patches available here. u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine. ZynqMP> part list usb 0

Partition Map for USB device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name Attributes
          Type GUID
          Partition GUID
    1     0x00000800      0x001007fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77 2     0x00100800      0x001197fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Quoted 7 lines

Do you have any idea why on one system is working fine to get to menu and on others there is an issue to get all partitions even u-boot is able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could be that the USB sub-system is simply not initialized yet when the boot manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough to run

     efidebug devices

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Quoted 8 lines
Quoted 2 lines

Block devices are enumerated in efi_disk_register(). Please, try to add debug output there to elucidate the problem.

I found where the problem is. First of all zcu102 didn't use the same image as others (it wasn't updated properly).
When you have CONFIG_EFI_CAPSULE_ON_DISK_EARLY that efi_disk_register() is called before usb block devices are detected and registered that's why grub doesn't see them.

Quoted 4 lines

The problem is CONFIG_EFI_SETUP_EARLY=y required by CONFIG_EFI_CAPSULE_ON_DISK_EARLY.

Why is USB initialized later then MMC?

It is not just usb. SCSI/sata are behaving in the same way too.

Quoted 4 lines

Overall we have a deficiency in the UEFI implementation in that we cannot deal with block devices added or removed after initialization.

Quoted 6 lines
Quoted 1 line

Here integration with the driver model is missing.

Right. And also there are commands which can create MBR partitions and I expect when you write image to SD and then run rescan or so you could get other partitions too.
Maybe hook via part_init()? with removing efi_disk_register.

Quoted 8 lines

For the record, I have proposed my ideas several times[1], [2]. I'm, however, no longer working on this issue as I have shifted my focus to UEFI secure boot and capsule update.

-Takahiro Akashi

[1] https://lists.denx.de/pipermail/u-boot/2018-November/347491.html
[2] https://lists.denx.de/pipermail/u-boot/2019-February/357923.html

I want to continue on this thread. I have disabled EFI_CAPSULE_ON_DISK_EARLY some time ago and trying to workaround that usb/scsi detection by simply calling usb reset and scsi reset as the part of PREBOOT. Then all disks are recorded and visible by grub.

But I found another issue which is kind of weird. We are using distroboot with soft of fixed sequence. Important part of sequence is sd, usb, scsi.

I have added grub on scsi and when I boot directly via run bootcmd_scsi0 everything is working fine. When I let distroboot to do the job it or run printenv -e before bootcmd_scsi0 I am getting exception. From debug it is visible that it is exception called from efi_disk_read_blocks.

    0  0x7ff5d188 hang()+20: include/bootstage.h, line 389
    1  0x7ff5f908 __assert_fail(): lib/panic.c, line 25
    2  0x7fe976a8 do_irq(): arch/arm/lib/interrupts_64.c, line 123
    3  0x7fe96a0c _restore_regs()+124: arch/arm/cpu/armv8/exceptions.S,
line 141
    4  0x7ff43740 efi_disk_read_blocks()+160: lib/efi_loader/efi_disk.c,

line 102

Logs are below and there is different place where disks are registered. Do you have any idea what could go wrong? Or something to check?

Thanks,
Michal

U-Boot 2021.10-rc1-00014-g81168a994f90 (Jul 29 2021 - 15:37:11 +0200)

Model: ZynqMP ZCU102 Rev1.0
Board: Xilinx ZynqMP
DRAM:  4 GiB
PMUFW:  v1.1
Xilinx I2C Legacy format at nvmem0:
 Board name:    zcu102
 Board rev:     1.0
 Board SN:      847316301727-67998
 Ethernet mac:  00:0a:35:03:70:f6
EL Level:       EL2
Chip ID:        zu9eg
WDT:   Not found!
NAND:  0 MiB
MMC:   mmc@ff170000: 0

Loading Environment from FAT... *** Error - No Valid Environment Area found *** Warning - bad env area, using default environment

In:    serial
Out:   serial
Err:   serial
Bootmode: LVL_SHFT_SD_MODE1
Reset reason:   SOFT

Net:
ZYNQ GEM: ff0e0000, mdio bus ff0e0000, phyaddr 12, interface rgmii-id
eth0: ethernet@ff0e0000

Reset SCSI
scanning bus for devices...
SATA link 0 timeout.
Target spinup took 0 ms.
AHCI 0001.0301 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq pm clo only pmp fbss pio slum part ccc apst
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
resetting USB...
Bus dwc3@fe200000: Register 2000440 NbrPorts 2
Starting the controller
USB XHCI 1.00
scanning bus dwc3@fe200000 for devices... 1 USB Device(s) found
scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot: 0
ZynqMP>
ZynqMP>
ZynqMP> run bootcmd_scsi0
scanning bus for devices...
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)

Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
... is now current device
Scanning scsi 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Scanning disk [email protected]...
Scanning disk ahci_scsi.id1lun0...
Found 5 disks
** Unable to read file ubootefi.var **
Failed to load EFI variables
Unable to find TPMv2 device
DFU alt info setting: done
BootOrder not defined
EFI boot manager: Cannot load any image
Found EFI removable media binary efi/boot/bootaa64.efi 647168 bytes read in 23 ms (26.8 MiB/s)
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Booting /efi\boot\bootaa64.efi
PE image measurement failed
Welcome to GRUB!

GNU GRUB version 2.11

all good here.

...

U-Boot 2021.10-rc1-00014-g81168a994f90 (Jul 29 2021 - 15:37:11 +0200)

Model: ZynqMP ZCU102 Rev1.0
Board: Xilinx ZynqMP
DRAM:  4 GiB
PMUFW:  v1.1
Xilinx I2C Legacy format at nvmem0:
 Board name:    zcu102
 Board rev:     1.0
 Board SN:      847316301727-67998
 Ethernet mac:  00:0a:35:03:70:f6
EL Level:       EL2
Chip ID:        zu9eg
WDT:   Not found!
NAND:  0 MiB
MMC:   mmc@ff170000: 0

Loading Environment from FAT... *** Error - No Valid Environment Area found *** Warning - bad env area, using default environment

In:    serial
Out:   serial
Err:   serial
Bootmode: LVL_SHFT_SD_MODE1
Reset reason:   SOFT

Net:
ZYNQ GEM: ff0e0000, mdio bus ff0e0000, phyaddr 12, interface rgmii-id
eth0: ethernet@ff0e0000

Reset SCSI
scanning bus for devices...
SATA link 0 timeout.
Target spinup took 0 ms.
AHCI 0001.0301 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq pm clo only pmp fbss pio slum part ccc apst
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
resetting USB...
Bus dwc3@fe200000: Register 2000440 NbrPorts 2
Starting the controller
USB XHCI 1.00
scanning bus dwc3@fe200000 for devices... 1 USB Device(s) found
scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot: 0
ZynqMP>
ZynqMP>
ZynqMP> print -e
Scanning disk [email protected]...
Scanning disk ahci_scsi.id1lun0...
Found 5 disks
** Unable to read file ubootefi.var **
Failed to load EFI variables
Unable to find TPMv2 device
DFU alt info setting: done
SecureBoot:

    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x1
    00000000: 00                                               .
SetupMode:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x1
    00000000: 01                                               .
AuditMode:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x1
    00000000: 00                                               .
DeployedMode:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x1
    00000000: 00                                               .
VendorKeys:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x1
    00000000: 00                                               .
PlatformLangCodes:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x6
    00000000: 65 6e 2d 55 53 00                                en-US.
PlatformLang:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    NV|BS|RT, DataSize = 0x6
    00000000: 65 6e 2d 55 53 00                                en-US.
OsIndicationsSupported:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x8
    00000000: 1c 00 00 00 00 00 00 00                          ........
SignatureSupport:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x20
    00000000: 26 16 c4 c1 4c 50 92 40 ac a9 41 f9 36 93 43 28
&[email protected](
    00000010: a1 59 c0 a5 e4 94 a7 4a 87 b5 ab 15 5c 2b f0 72
.Y.....J....\+.r
OsIndications:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    NV|BS|RT, DataSize = 0x8
    00000000: 00 00 00 00 00 00 00 00                          ........

ZynqMP> run bootcmd_scsi0
scanning bus for devices...
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)

Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
... is now current device
Scanning scsi 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC BootOrder not defined
EFI boot manager: Cannot load any image
Found EFI removable media binary efi/boot/bootaa64.efi 647168 bytes read in 24 ms (25.7 MiB/s)
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Booting /efi\boot\bootaa64.efi
PE image measurement failed
Welcome to GRUB!

"Synchronous Abort" handler, esr 0x02000000
elr: ffffffffa816c5b0 lr : 000000000805e218 (reloc)
elr: 00000000200005b0 lr : 000000007fef2218 x0 : 0000000000000020 x1 : 0000000000000000 x2 : 0000000000000040 x3 : 000000004ff49400 x4 : 00000000200005b0 x5 : 000000007be4abb0 x6 : 000000007ffa5230 x7 : 000000007ffa5230 x8 : 0000000000000006 x9 : fffffffffffffff0
x10: 0000000000000001 x11: 000000007be4abb0
x12: 0000000000000040 x13: 0000000000000200
x14: 000000007fe95208 x15: 0000000000000000
x16: 0000000077d4f8d0 x17: 0000000000000000
x18: 000000007be13da0 x19: 000000007be22340
x20: 0000000000000020 x21: 0000000000000040
x22: 0000000000000000 x23: 000000004ff49400
x24: 0000000000000000 x25: 0000000000000100
x26: 000000000000000f x27: 00000000000001ff
x28: 000000007be4cb40 x29: 000000007be06ec0

Code: 000165fa 0b2d05de 0000ffff 00000000 (20000590)

UEFI image [0x0000000077d48000:0x0000000077de5fff] '/efi\boot\bootaa64.efi' Resetting CPU ...

### ERROR ### Please RESET the board ###

#7 lore raw
Quoted 155 lines · Michal Simek

Hi,

Quoted 128 lines · AKASHI Takahiro
Quoted 120 lines · Michal Simek
Quoted 106 lines · Heinrich Schuchardt
Quoted 96 lines · Michal Simek

Hi,

Quoted 93 lines · Heinrich Schuchardt
Quoted 11 lines · Michal Simek

Hi,

I am playing with booting from USB via EFI. And I see very weird behavior. I have burnt image with grub to USB flashdisk and I have tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board. On zcu102 grub is going to boot menu and everything is working fine as expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

Quoted 5 lines

On zcu102(working board) I also see usb(gpt) partitions and SD. grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

Quoted 3 lines

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This will not stop disk enumeration.

Quoted 35 lines

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they are
the same. SOM doesn't have usb enabled by default (but I enabled it) but
grub starts which means that communication with USB is fine.

It is based on my latest patches available here. u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine. ZynqMP> part list usb 0

Partition Map for USB device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name Attributes
          Type GUID
          Partition GUID
    1     0x00000800      0x001007fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77 2     0x00100800      0x001197fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Quoted 7 lines

Do you have any idea why on one system is working fine to get to menu and on others there is an issue to get all partitions even u-boot is able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could be that the USB sub-system is simply not initialized yet when the boot manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough to run

     efidebug devices

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Quoted 8 lines
Quoted 2 lines

Block devices are enumerated in efi_disk_register(). Please, try to add debug output there to elucidate the problem.

I found where the problem is. First of all zcu102 didn't use the same image as others (it wasn't updated properly).
When you have CONFIG_EFI_CAPSULE_ON_DISK_EARLY that efi_disk_register() is called before usb block devices are detected and registered that's why grub doesn't see them.

Quoted 4 lines

The problem is CONFIG_EFI_SETUP_EARLY=y required by CONFIG_EFI_CAPSULE_ON_DISK_EARLY.

Why is USB initialized later then MMC?

It is not just usb. SCSI/sata are behaving in the same way too.

Quoted 4 lines

Overall we have a deficiency in the UEFI implementation in that we cannot deal with block devices added or removed after initialization.

Quoted 6 lines
Quoted 1 line

Here integration with the driver model is missing.

Right. And also there are commands which can create MBR partitions and I expect when you write image to SD and then run rescan or so you could get other partitions too.
Maybe hook via part_init()? with removing efi_disk_register.

Quoted 8 lines

For the record, I have proposed my ideas several times[1], [2]. I'm, however, no longer working on this issue as I have shifted my focus to UEFI secure boot and capsule update.

-Takahiro Akashi

[1] https://lists.denx.de/pipermail/u-boot/2018-November/347491.html
[2] https://lists.denx.de/pipermail/u-boot/2019-February/357923.html

I want to continue on this thread. I have disabled EFI_CAPSULE_ON_DISK_EARLY some time ago and trying to workaround that usb/scsi detection by simply calling usb reset and scsi reset as the part of PREBOOT. Then all disks are recorded and visible by grub.

But I found another issue which is kind of weird. We are using distroboot with soft of fixed sequence. Important part of sequence is sd, usb, scsi.

I have added grub on scsi and when I boot directly via run bootcmd_scsi0 everything is working fine. When I let distroboot to do the job it or run printenv -e before bootcmd_scsi0 I am getting exception. From debug it is visible that it is exception called from efi_disk_read_blocks.

Quoted 7 lines
    0  0x7ff5d188 hang()+20: include/bootstage.h, line 389
    1  0x7ff5f908 __assert_fail(): lib/panic.c, line 25
    2  0x7fe976a8 do_irq(): arch/arm/lib/interrupts_64.c, line 123
    3  0x7fe96a0c _restore_regs()+124: arch/arm/cpu/armv8/exceptions.S,
line 141
    4  0x7ff43740 efi_disk_read_blocks()+160: lib/efi_loader/efi_disk.c,

line 102

How and when did you get this stack trace?

-Takahiro Akashi

Quoted 233 lines

Logs are below and there is different place where disks are registered. Do you have any idea what could go wrong? Or something to check?

Thanks,
Michal

U-Boot 2021.10-rc1-00014-g81168a994f90 (Jul 29 2021 - 15:37:11 +0200)

Model: ZynqMP ZCU102 Rev1.0
Board: Xilinx ZynqMP
DRAM:  4 GiB
PMUFW:  v1.1
Xilinx I2C Legacy format at nvmem0:
 Board name:    zcu102
 Board rev:     1.0
 Board SN:      847316301727-67998
 Ethernet mac:  00:0a:35:03:70:f6
EL Level:       EL2
Chip ID:        zu9eg
WDT:   Not found!
NAND:  0 MiB
MMC:   mmc@ff170000: 0

Loading Environment from FAT... *** Error - No Valid Environment Area found *** Warning - bad env area, using default environment

In:    serial
Out:   serial
Err:   serial
Bootmode: LVL_SHFT_SD_MODE1
Reset reason:   SOFT

Net:
ZYNQ GEM: ff0e0000, mdio bus ff0e0000, phyaddr 12, interface rgmii-id
eth0: ethernet@ff0e0000

Reset SCSI
scanning bus for devices...
SATA link 0 timeout.
Target spinup took 0 ms.
AHCI 0001.0301 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq pm clo only pmp fbss pio slum part ccc apst
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
resetting USB...
Bus dwc3@fe200000: Register 2000440 NbrPorts 2
Starting the controller
USB XHCI 1.00
scanning bus dwc3@fe200000 for devices... 1 USB Device(s) found
scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot: 0
ZynqMP>
ZynqMP>
ZynqMP> run bootcmd_scsi0
scanning bus for devices...
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)

Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
... is now current device
Scanning scsi 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Scanning disk [email protected]...
Scanning disk ahci_scsi.id1lun0...
Found 5 disks
** Unable to read file ubootefi.var **
Failed to load EFI variables
Unable to find TPMv2 device
DFU alt info setting: done
BootOrder not defined
EFI boot manager: Cannot load any image
Found EFI removable media binary efi/boot/bootaa64.efi 647168 bytes read in 23 ms (26.8 MiB/s)
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Booting /efi\boot\bootaa64.efi
PE image measurement failed
Welcome to GRUB!

GNU GRUB version 2.11

all good here.

...

U-Boot 2021.10-rc1-00014-g81168a994f90 (Jul 29 2021 - 15:37:11 +0200)

Model: ZynqMP ZCU102 Rev1.0
Board: Xilinx ZynqMP
DRAM:  4 GiB
PMUFW:  v1.1
Xilinx I2C Legacy format at nvmem0:
 Board name:    zcu102
 Board rev:     1.0
 Board SN:      847316301727-67998
 Ethernet mac:  00:0a:35:03:70:f6
EL Level:       EL2
Chip ID:        zu9eg
WDT:   Not found!
NAND:  0 MiB
MMC:   mmc@ff170000: 0

Loading Environment from FAT... *** Error - No Valid Environment Area found *** Warning - bad env area, using default environment

In:    serial
Out:   serial
Err:   serial
Bootmode: LVL_SHFT_SD_MODE1
Reset reason:   SOFT

Net:
ZYNQ GEM: ff0e0000, mdio bus ff0e0000, phyaddr 12, interface rgmii-id
eth0: ethernet@ff0e0000

Reset SCSI
scanning bus for devices...
SATA link 0 timeout.
Target spinup took 0 ms.
AHCI 0001.0301 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq pm clo only pmp fbss pio slum part ccc apst
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
resetting USB...
Bus dwc3@fe200000: Register 2000440 NbrPorts 2
Starting the controller
USB XHCI 1.00
scanning bus dwc3@fe200000 for devices... 1 USB Device(s) found
scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot: 0
ZynqMP>
ZynqMP>
ZynqMP> print -e
Scanning disk [email protected]...
Scanning disk ahci_scsi.id1lun0...
Found 5 disks
** Unable to read file ubootefi.var **
Failed to load EFI variables
Unable to find TPMv2 device
DFU alt info setting: done
SecureBoot:

    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x1
    00000000: 00                                               .
SetupMode:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x1
    00000000: 01                                               .
AuditMode:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x1
    00000000: 00                                               .
DeployedMode:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x1
    00000000: 00                                               .
VendorKeys:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x1
    00000000: 00                                               .
PlatformLangCodes:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x6
    00000000: 65 6e 2d 55 53 00                                en-US.
PlatformLang:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    NV|BS|RT, DataSize = 0x6
    00000000: 65 6e 2d 55 53 00                                en-US.
OsIndicationsSupported:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x8
    00000000: 1c 00 00 00 00 00 00 00                          ........
SignatureSupport:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    BS|RT|RO, DataSize = 0x20
    00000000: 26 16 c4 c1 4c 50 92 40 ac a9 41 f9 36 93 43 28
&[email protected](
    00000010: a1 59 c0 a5 e4 94 a7 4a 87 b5 ab 15 5c 2b f0 72
.Y.....J....\+.r
OsIndications:
    8be4df61-93ca-11d2-aa0d-00e098032b8c EFI_GLOBAL_VARIABLE_GUID
    NV|BS|RT, DataSize = 0x8
    00000000: 00 00 00 00 00 00 00 00                          ........

ZynqMP> run bootcmd_scsi0
scanning bus for devices...
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)

Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
... is now current device
Scanning scsi 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC BootOrder not defined
EFI boot manager: Cannot load any image
Found EFI removable media binary efi/boot/bootaa64.efi 647168 bytes read in 24 ms (25.7 MiB/s)
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Booting /efi\boot\bootaa64.efi
PE image measurement failed
Welcome to GRUB!

"Synchronous Abort" handler, esr 0x02000000
elr: ffffffffa816c5b0 lr : 000000000805e218 (reloc)
elr: 00000000200005b0 lr : 000000007fef2218 x0 : 0000000000000020 x1 : 0000000000000000 x2 : 0000000000000040 x3 : 000000004ff49400 x4 : 00000000200005b0 x5 : 000000007be4abb0 x6 : 000000007ffa5230 x7 : 000000007ffa5230 x8 : 0000000000000006 x9 : fffffffffffffff0
x10: 0000000000000001 x11: 000000007be4abb0
x12: 0000000000000040 x13: 0000000000000200
x14: 000000007fe95208 x15: 0000000000000000
x16: 0000000077d4f8d0 x17: 0000000000000000
x18: 000000007be13da0 x19: 000000007be22340
x20: 0000000000000020 x21: 0000000000000040
x22: 0000000000000000 x23: 000000004ff49400
x24: 0000000000000000 x25: 0000000000000100
x26: 000000000000000f x27: 00000000000001ff
x28: 000000007be4cb40 x29: 000000007be06ec0

Code: 000165fa 0b2d05de 0000ffff 00000000 (20000590)

UEFI image [0x0000000077d48000:0x0000000077de5fff] '/efi\boot\bootaa64.efi' Resetting CPU ...

### ERROR ### Please RESET the board ###

#8 lore raw
Quoted 164 lines · AKASHI Takahiro
Quoted 155 lines · Michal Simek

Hi,

Quoted 128 lines · AKASHI Takahiro
Quoted 120 lines · Michal Simek
Quoted 106 lines · Heinrich Schuchardt
Quoted 96 lines · Michal Simek

Hi,

Quoted 93 lines · Heinrich Schuchardt
Quoted 11 lines · Michal Simek

Hi,

I am playing with booting from USB via EFI. And I see very weird behavior. I have burnt image with grub to USB flashdisk and I have tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board. On zcu102 grub is going to boot menu and everything is working fine as expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

Quoted 5 lines

On zcu102(working board) I also see usb(gpt) partitions and SD. grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

Quoted 3 lines

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This will not stop disk enumeration.

Quoted 35 lines

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they are
the same. SOM doesn't have usb enabled by default (but I enabled it) but
grub starts which means that communication with USB is fine.

It is based on my latest patches available here. u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine. ZynqMP> part list usb 0

Partition Map for USB device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name Attributes
          Type GUID
          Partition GUID
    1     0x00000800      0x001007fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77 2     0x00100800      0x001197fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Quoted 7 lines

Do you have any idea why on one system is working fine to get to menu and on others there is an issue to get all partitions even u-boot is able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could be that the USB sub-system is simply not initialized yet when the boot manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough to run

     efidebug devices

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Quoted 8 lines
Quoted 2 lines

Block devices are enumerated in efi_disk_register(). Please, try to add debug output there to elucidate the problem.

I found where the problem is. First of all zcu102 didn't use the same image as others (it wasn't updated properly).
When you have CONFIG_EFI_CAPSULE_ON_DISK_EARLY that efi_disk_register() is called before usb block devices are detected and registered that's why grub doesn't see them.

Quoted 4 lines

The problem is CONFIG_EFI_SETUP_EARLY=y required by CONFIG_EFI_CAPSULE_ON_DISK_EARLY.

Why is USB initialized later then MMC?

It is not just usb. SCSI/sata are behaving in the same way too.

Quoted 4 lines

Overall we have a deficiency in the UEFI implementation in that we cannot deal with block devices added or removed after initialization.

Quoted 6 lines
Quoted 1 line

Here integration with the driver model is missing.

Right. And also there are commands which can create MBR partitions and I expect when you write image to SD and then run rescan or so you could get other partitions too.
Maybe hook via part_init()? with removing efi_disk_register.

Quoted 8 lines

For the record, I have proposed my ideas several times[1], [2]. I'm, however, no longer working on this issue as I have shifted my focus to UEFI secure boot and capsule update.

-Takahiro Akashi

[1] https://lists.denx.de/pipermail/u-boot/2018-November/347491.html
[2] https://lists.denx.de/pipermail/u-boot/2019-February/357923.html

I want to continue on this thread. I have disabled EFI_CAPSULE_ON_DISK_EARLY some time ago and trying to workaround that usb/scsi detection by simply calling usb reset and scsi reset as the part of PREBOOT. Then all disks are recorded and visible by grub.

But I found another issue which is kind of weird. We are using distroboot with soft of fixed sequence. Important part of sequence is sd, usb, scsi.

I have added grub on scsi and when I boot directly via run bootcmd_scsi0 everything is working fine. When I let distroboot to do the job it or run printenv -e before bootcmd_scsi0 I am getting exception. From debug it is visible that it is exception called from efi_disk_read_blocks.

Quoted 7 lines
    0  0x7ff5d188 hang()+20: include/bootstage.h, line 389
    1  0x7ff5f908 __assert_fail(): lib/panic.c, line 25
    2  0x7fe976a8 do_irq(): arch/arm/lib/interrupts_64.c, line 123
    3  0x7fe96a0c _restore_regs()+124: arch/arm/cpu/armv8/exceptions.S,
line 141
    4  0x7ff43740 efi_disk_read_blocks()+160: lib/efi_loader/efi_disk.c,

line 102

Quoted 1 line

How and when did you get this stack trace?

When Abort happened I connected Xilinx debugger via jtag and look at cpu backtrace.

Thanks,
Michal

#9 lore raw
Quoted 167 lines · Michal Simek
Quoted 164 lines · AKASHI Takahiro
Quoted 162 lines · Michal Simek

Hi,

Quoted 128 lines · AKASHI Takahiro
Quoted 120 lines · Michal Simek
Quoted 106 lines · Heinrich Schuchardt
Quoted 96 lines · Michal Simek

Hi,

Quoted 93 lines · Heinrich Schuchardt
Quoted 11 lines · Michal Simek

Hi,

I am playing with booting from USB via EFI. And I see very weird behavior. I have burnt image with grub to USB flashdisk and I have tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board. On zcu102 grub is going to boot menu and everything is working fine as expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

Quoted 5 lines

On zcu102(working board) I also see usb(gpt) partitions and SD. grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

Quoted 3 lines

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This will not stop disk enumeration.

Quoted 35 lines

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they are
the same. SOM doesn't have usb enabled by default (but I enabled it) but
grub starts which means that communication with USB is fine.

It is based on my latest patches available here. u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine. ZynqMP> part list usb 0

Partition Map for USB device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name Attributes
          Type GUID
          Partition GUID
    1     0x00000800      0x001007fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77 2     0x00100800      0x001197fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Quoted 7 lines

Do you have any idea why on one system is working fine to get to menu and on others there is an issue to get all partitions even u-boot is able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could be that the USB sub-system is simply not initialized yet when the boot manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough to run

     efidebug devices

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Quoted 8 lines
Quoted 2 lines

Block devices are enumerated in efi_disk_register(). Please, try to add debug output there to elucidate the problem.

I found where the problem is. First of all zcu102 didn't use the same image as others (it wasn't updated properly).
When you have CONFIG_EFI_CAPSULE_ON_DISK_EARLY that efi_disk_register() is called before usb block devices are detected and registered that's why grub doesn't see them.

Quoted 4 lines

The problem is CONFIG_EFI_SETUP_EARLY=y required by CONFIG_EFI_CAPSULE_ON_DISK_EARLY.

Why is USB initialized later then MMC?

It is not just usb. SCSI/sata are behaving in the same way too.

Quoted 4 lines

Overall we have a deficiency in the UEFI implementation in that we cannot deal with block devices added or removed after initialization.

Quoted 6 lines
Quoted 1 line

Here integration with the driver model is missing.

Right. And also there are commands which can create MBR partitions and I expect when you write image to SD and then run rescan or so you could get other partitions too.
Maybe hook via part_init()? with removing efi_disk_register.

Quoted 8 lines

For the record, I have proposed my ideas several times[1], [2]. I'm, however, no longer working on this issue as I have shifted my focus to UEFI secure boot and capsule update.

-Takahiro Akashi

[1] https://lists.denx.de/pipermail/u-boot/2018-November/347491.html
[2] https://lists.denx.de/pipermail/u-boot/2019-February/357923.html

I want to continue on this thread. I have disabled EFI_CAPSULE_ON_DISK_EARLY some time ago and trying to workaround that usb/scsi detection by simply calling usb reset and scsi reset as the part of PREBOOT. Then all disks are recorded and visible by grub.

But I found another issue which is kind of weird. We are using distroboot with soft of fixed sequence. Important part of sequence is sd, usb, scsi.

I have added grub on scsi and when I boot directly via run bootcmd_scsi0 everything is working fine. When I let distroboot to do the job it or run printenv -e before bootcmd_scsi0 I am getting exception. From debug it is visible that it is exception called from efi_disk_read_blocks.

    0  0x7ff5d188 hang()+20: include/bootstage.h, line 389
    1  0x7ff5f908 __assert_fail(): lib/panic.c, line 25
    2  0x7fe976a8 do_irq(): arch/arm/lib/interrupts_64.c, line 123
    3  0x7fe96a0c _restore_regs()+124: arch/arm/cpu/armv8/exceptions.S,
line 141
    4  0x7ff43740 efi_disk_read_blocks()+160: lib/efi_loader/efi_disk.c,

line 102

Quoted 4 lines
Quoted 1 line

How and when did you get this stack trace?

When Abort happened I connected Xilinx debugger via jtag and look at cpu backtrace.

OK, but we are already in grub here and such a trace (in U-Boot) doesn't make sense. Right?

-Takahiro Akashi

Quoted 3 lines

Thanks,
Michal

#10 lore raw
Quoted 168 lines · AKASHI Takahiro
Quoted 167 lines · Michal Simek
Quoted 164 lines · AKASHI Takahiro
Quoted 162 lines · Michal Simek

Hi,

Quoted 128 lines · AKASHI Takahiro
Quoted 120 lines · Michal Simek
Quoted 106 lines · Heinrich Schuchardt
Quoted 96 lines · Michal Simek

Hi,

Quoted 93 lines · Heinrich Schuchardt
Quoted 11 lines · Michal Simek

Hi,

I am playing with booting from USB via EFI. And I see very weird behavior. I have burnt image with grub to USB flashdisk and I have tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board. On zcu102 grub is going to boot menu and everything is working fine as expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

Quoted 5 lines

On zcu102(working board) I also see usb(gpt) partitions and SD. grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

Quoted 3 lines

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This will not stop disk enumeration.

Quoted 35 lines

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they are
the same. SOM doesn't have usb enabled by default (but I enabled it) but
grub starts which means that communication with USB is fine.

It is based on my latest patches available here. u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine. ZynqMP> part list usb 0

Partition Map for USB device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name Attributes
          Type GUID
          Partition GUID
    1     0x00000800      0x001007fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77 2     0x00100800      0x001197fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Quoted 7 lines

Do you have any idea why on one system is working fine to get to menu and on others there is an issue to get all partitions even u-boot is able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could be that the USB sub-system is simply not initialized yet when the boot manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough to run

     efidebug devices

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Quoted 8 lines
Quoted 2 lines

Block devices are enumerated in efi_disk_register(). Please, try to add debug output there to elucidate the problem.

I found where the problem is. First of all zcu102 didn't use the same image as others (it wasn't updated properly).
When you have CONFIG_EFI_CAPSULE_ON_DISK_EARLY that efi_disk_register() is called before usb block devices are detected and registered that's why grub doesn't see them.

Quoted 4 lines

The problem is CONFIG_EFI_SETUP_EARLY=y required by CONFIG_EFI_CAPSULE_ON_DISK_EARLY.

Why is USB initialized later then MMC?

It is not just usb. SCSI/sata are behaving in the same way too.

Quoted 4 lines

Overall we have a deficiency in the UEFI implementation in that we cannot deal with block devices added or removed after initialization.

Quoted 6 lines
Quoted 1 line

Here integration with the driver model is missing.

Right. And also there are commands which can create MBR partitions and I expect when you write image to SD and then run rescan or so you could get other partitions too.
Maybe hook via part_init()? with removing efi_disk_register.

Quoted 8 lines

For the record, I have proposed my ideas several times[1], [2]. I'm, however, no longer working on this issue as I have shifted my focus to UEFI secure boot and capsule update.

-Takahiro Akashi

[1] https://lists.denx.de/pipermail/u-boot/2018-November/347491.html
[2] https://lists.denx.de/pipermail/u-boot/2019-February/357923.html

I want to continue on this thread. I have disabled EFI_CAPSULE_ON_DISK_EARLY some time ago and trying to workaround that usb/scsi detection by simply calling usb reset and scsi reset as the part of PREBOOT. Then all disks are recorded and visible by grub.

But I found another issue which is kind of weird. We are using distroboot with soft of fixed sequence. Important part of sequence is sd, usb, scsi.

I have added grub on scsi and when I boot directly via run bootcmd_scsi0 everything is working fine. When I let distroboot to do the job it or run printenv -e before bootcmd_scsi0 I am getting exception. From debug it is visible that it is exception called from efi_disk_read_blocks.

    0  0x7ff5d188 hang()+20: include/bootstage.h, line 389
    1  0x7ff5f908 __assert_fail(): lib/panic.c, line 25
    2  0x7fe976a8 do_irq(): arch/arm/lib/interrupts_64.c, line 123
    3  0x7fe96a0c _restore_regs()+124: arch/arm/cpu/armv8/exceptions.S,
line 141
    4  0x7ff43740 efi_disk_read_blocks()+160: lib/efi_loader/efi_disk.c,

line 102

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

How and when did you get this stack trace?

When Abort happened I connected Xilinx debugger via jtag and look at cpu backtrace.

OK, but we are already in grub here and such a trace (in U-Boot) doesn't make sense. Right?

Correct grub already started. But I expect it is still using U-Boot drivers and all exception handlers are still in place from u-boot.

Maybe it is just sata/scsi related issue in EFI but weird is that when disks are scan just before command everything is working fine.

Should I try to initialize and populate EFI object list all the time? (Remove this code for testing)

	/* Initialize once only */
	if (efi_obj_list_initialized != OBJ_LIST_NOT_INITIALIZED)
		return efi_obj_list_initialized;

Or based on this thread to try your series pointed above?

Thanks,
Michal

#11 lore raw

Hi Michal
Apologies for my late reply, I was on vacation!

[...]

Quoted 9 lines
Quoted 6 lines
Quoted 3 lines

When Abort happened I connected Xilinx debugger via jtag and look at cpu backtrace.

OK, but we are already in grub here and such a trace (in U-Boot) doesn't make sense. Right?

Correct grub already started. But I expect it is still using U-Boot drivers and all exception handlers are still in place from u-boot.

Yes, U-Boot is still alive and GRUB is accessing some of the functionality via Boottime services.

Quoted 11 lines

Maybe it is just sata/scsi related issue in EFI but weird is that when disks are scan just before command everything is working fine.

Should I try to initialize and populate EFI object list all the time? (Remove this code for testing)

        /* Initialize once only */
        if (efi_obj_list_initialized != OBJ_LIST_NOT_INITIALIZED)
                return efi_obj_list_initialized;

Or based on this thread to try your series pointed above?

This smells like a different init sequence when a command is issued first. I'll try having a closer look. Let me know if you find anything else

Cheers
/Ilias

Quoted 4 lines

Thanks,
Michal

#12 lore raw

Hi Ilias,

Quoted 2 lines · Ilias Apalodimas

Hi Michal
Apologies for my late reply, I was on vacation!

no problem at all.

Quoted 29 lines

[...]

Quoted 9 lines
Quoted 6 lines
Quoted 3 lines

When Abort happened I connected Xilinx debugger via jtag and look at cpu backtrace.

OK, but we are already in grub here and such a trace (in U-Boot) doesn't make sense. Right?

Correct grub already started. But I expect it is still using U-Boot drivers and all exception handlers are still in place from u-boot.

Yes, U-Boot is still alive and GRUB is accessing some of the functionality via Boottime services.

Quoted 10 lines

Maybe it is just sata/scsi related issue in EFI but weird is that when disks are scan just before command everything is working fine.

Should I try to initialize and populate EFI object list all the time? (Remove this code for testing)

        /* Initialize once only */
        if (efi_obj_list_initialized != OBJ_LIST_NOT_INITIALIZED)
                return efi_obj_list_initialized;
Quoted 4 lines
Quoted 1 line

Or based on this thread to try your series pointed above?

This smells like a different init sequence when a command is issued first. I'll try having a closer look. Let me know if you find anything else

Did you have a time to look at it?

Thanks,
Michal

#13 lore raw
In reply to Michal Simek
Quoted 173 lines · Michal Simek
Quoted 170 lines · AKASHI Takahiro
Quoted 169 lines · Michal Simek
Quoted 165 lines · AKASHI Takahiro
Quoted 155 lines · Michal Simek

Hi,

Quoted 128 lines · AKASHI Takahiro
Quoted 120 lines · Michal Simek
Quoted 106 lines · Heinrich Schuchardt
Quoted 96 lines · Michal Simek

Hi,

Quoted 93 lines · Heinrich Schuchardt
Quoted 11 lines · Michal Simek

Hi,

I am playing with booting from USB via EFI. And I see very weird behavior. I have burnt image with grub to USB flashdisk and I have tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board. On zcu102 grub is going to boot menu and everything is working fine as expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

Quoted 5 lines

On zcu102(working board) I also see usb(gpt) partitions and SD. grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

Quoted 3 lines

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This will not stop disk enumeration.

Quoted 35 lines

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they are
the same. SOM doesn't have usb enabled by default (but I enabled it) but
grub starts which means that communication with USB is fine.

It is based on my latest patches available here. u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine. ZynqMP> part list usb 0

Partition Map for USB device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name Attributes
          Type GUID
          Partition GUID
    1     0x00000800      0x001007fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77 2     0x00100800      0x001197fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Quoted 7 lines

Do you have any idea why on one system is working fine to get to menu and on others there is an issue to get all partitions even u-boot is able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could be that the USB sub-system is simply not initialized yet when the boot manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough to run

     efidebug devices

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Quoted 8 lines
Quoted 2 lines

Block devices are enumerated in efi_disk_register(). Please, try to add debug output there to elucidate the problem.

I found where the problem is. First of all zcu102 didn't use the same image as others (it wasn't updated properly).
When you have CONFIG_EFI_CAPSULE_ON_DISK_EARLY that efi_disk_register() is called before usb block devices are detected and registered that's why grub doesn't see them.

Quoted 4 lines

The problem is CONFIG_EFI_SETUP_EARLY=y required by CONFIG_EFI_CAPSULE_ON_DISK_EARLY.

Why is USB initialized later then MMC?

It is not just usb. SCSI/sata are behaving in the same way too.

Quoted 4 lines

Overall we have a deficiency in the UEFI implementation in that we cannot deal with block devices added or removed after initialization.

Quoted 6 lines
Quoted 1 line

Here integration with the driver model is missing.

Right. And also there are commands which can create MBR partitions and I expect when you write image to SD and then run rescan or so you could get other partitions too.
Maybe hook via part_init()? with removing efi_disk_register.

Quoted 8 lines

For the record, I have proposed my ideas several times[1], [2]. I'm, however, no longer working on this issue as I have shifted my focus to UEFI secure boot and capsule update.

-Takahiro Akashi

[1] https://lists.denx.de/pipermail/u-boot/2018-November/347491.html
[2] https://lists.denx.de/pipermail/u-boot/2019-February/357923.html

I want to continue on this thread. I have disabled EFI_CAPSULE_ON_DISK_EARLY some time ago and trying to workaround that usb/scsi detection by simply calling usb reset and scsi reset as the part of PREBOOT. Then all disks are recorded and visible by grub.

But I found another issue which is kind of weird. We are using distroboot with soft of fixed sequence. Important part of sequence is sd, usb, scsi.

I have added grub on scsi and when I boot directly via run bootcmd_scsi0 everything is working fine. When I let distroboot to do the job it or run printenv -e before bootcmd_scsi0 I am getting exception. From debug it is visible that it is exception called from efi_disk_read_blocks.

Quoted 7 lines
    0  0x7ff5d188 hang()+20: include/bootstage.h, line 389
    1  0x7ff5f908 __assert_fail(): lib/panic.c, line 25
    2  0x7fe976a8 do_irq(): arch/arm/lib/interrupts_64.c, line 123
    3  0x7fe96a0c _restore_regs()+124: arch/arm/cpu/armv8/exceptions.S,
line 141
    4  0x7ff43740 efi_disk_read_blocks()+160: lib/efi_loader/efi_disk.c,

line 102

How and when did you get this stack trace?

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

When Abort happened I connected Xilinx debugger via jtag and look at cpu backtrace.

OK, but we are already in grub here and such a trace (in U-Boot) doesn't make sense. Right?

Correct grub already started. But I expect it is still using U-Boot drivers and all exception handlers are still in place from u-boot.

Yeah, but what I didn't understand was:

!"Synchronous Abort" handler, esr 0x02000000
!elr: ffffffffa816c5b0 lr : 000000000805e218 (reloc)
!elr: 00000000200005b0 lr : 000000007fef2218
(snip)
!Code: 000165fa 0b2d05de 0000ffff 00000000 (20000590)
!UEFI image [0x0000000077d48000:0x0000000077de5fff] '/efi\boot\bootaa64.efi'

"Code:" at the exception doesn't seem to be sane assembler, and "elr" is not within the code of neither U-Boot nor shim/grub(bootaa64.efi). ("esr" doesn't tell us anything.)
So I wondered where the backtrace came from.

BTW, can you please confirm which function sits at the address of "lr" (=0x7fe2218)?

Quoted 2 lines

Maybe it is just sata/scsi related issue in EFI but weird is that when disks are scan just before command everything is working fine.

What do you mean by "when disks are scanned just before the command"? The case when you ran "run bootcmd_scsi" without "printenv -e"?

Do you reproduce the problem even if you revert the patch, "xilinx: zynqmp: Initialize usb and scsi via preboot", and run the commands, "run scsi_init; [printenv -e;] run bootcmd_scsi?

Can you also try other EFI commands, like "efidebug devices"?

Quoted 5 lines

Should I try to initialize and populate EFI object list all the time? (Remove this code for testing)

	/* Initialize once only */
	if (efi_obj_list_initialized != OBJ_LIST_NOT_INITIALIZED)
		return efi_obj_list_initialized;

No. EFI subsystem is not implemented in that way. efi_obj_list_initialized() should be called only once, and it will be called at the first invocation of any EFI-related commands, bootefi or printenv -e (or efidebug).

-Takahiro Akashi

Quoted 5 lines

Or based on this thread to try your series pointed above?

Thanks,
Michal

#14 lore raw
Quoted 192 lines · AKASHI Takahiro
Quoted 173 lines · Michal Simek
Quoted 170 lines · AKASHI Takahiro
Quoted 169 lines · Michal Simek
Quoted 165 lines · AKASHI Takahiro
Quoted 155 lines · Michal Simek

Hi,

Quoted 128 lines · AKASHI Takahiro
Quoted 120 lines · Michal Simek
Quoted 106 lines · Heinrich Schuchardt
Quoted 96 lines · Michal Simek

Hi,

Quoted 93 lines · Heinrich Schuchardt
Quoted 11 lines · Michal Simek

Hi,

I am playing with booting from USB via EFI. And I see very weird behavior. I have burnt image with grub to USB flashdisk and I have tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board. On zcu102 grub is going to boot menu and everything is working fine as expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

Quoted 5 lines

On zcu102(working board) I also see usb(gpt) partitions and SD. grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

Quoted 3 lines

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This will not stop disk enumeration.

Quoted 35 lines

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they are
the same. SOM doesn't have usb enabled by default (but I enabled it) but
grub starts which means that communication with USB is fine.

It is based on my latest patches available here. u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine. ZynqMP> part list usb 0

Partition Map for USB device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name Attributes
          Type GUID
          Partition GUID
    1     0x00000800      0x001007fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77 2     0x00100800      0x001197fe      "Microsoft basic data" attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7 type:   data
          guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Quoted 7 lines

Do you have any idea why on one system is working fine to get to menu and on others there is an issue to get all partitions even u-boot is able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could be that the USB sub-system is simply not initialized yet when the boot manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough to run

     efidebug devices

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Quoted 8 lines
Quoted 2 lines

Block devices are enumerated in efi_disk_register(). Please, try to add debug output there to elucidate the problem.

I found where the problem is. First of all zcu102 didn't use the same image as others (it wasn't updated properly).
When you have CONFIG_EFI_CAPSULE_ON_DISK_EARLY that efi_disk_register() is called before usb block devices are detected and registered that's why grub doesn't see them.

Quoted 4 lines

The problem is CONFIG_EFI_SETUP_EARLY=y required by CONFIG_EFI_CAPSULE_ON_DISK_EARLY.

Why is USB initialized later then MMC?

It is not just usb. SCSI/sata are behaving in the same way too.

Quoted 4 lines

Overall we have a deficiency in the UEFI implementation in that we cannot deal with block devices added or removed after initialization.

Quoted 6 lines
Quoted 1 line

Here integration with the driver model is missing.

Right. And also there are commands which can create MBR partitions and I expect when you write image to SD and then run rescan or so you could get other partitions too.
Maybe hook via part_init()? with removing efi_disk_register.

Quoted 8 lines

For the record, I have proposed my ideas several times[1], [2]. I'm, however, no longer working on this issue as I have shifted my focus to UEFI secure boot and capsule update.

-Takahiro Akashi

[1] https://lists.denx.de/pipermail/u-boot/2018-November/347491.html
[2] https://lists.denx.de/pipermail/u-boot/2019-February/357923.html

I want to continue on this thread. I have disabled EFI_CAPSULE_ON_DISK_EARLY some time ago and trying to workaround that usb/scsi detection by simply calling usb reset and scsi reset as the part of PREBOOT. Then all disks are recorded and visible by grub.

But I found another issue which is kind of weird. We are using distroboot with soft of fixed sequence. Important part of sequence is sd, usb, scsi.

I have added grub on scsi and when I boot directly via run bootcmd_scsi0 everything is working fine. When I let distroboot to do the job it or run printenv -e before bootcmd_scsi0 I am getting exception. From debug it is visible that it is exception called from efi_disk_read_blocks.

Quoted 7 lines
    0  0x7ff5d188 hang()+20: include/bootstage.h, line 389
    1  0x7ff5f908 __assert_fail(): lib/panic.c, line 25
    2  0x7fe976a8 do_irq(): arch/arm/lib/interrupts_64.c, line 123
    3  0x7fe96a0c _restore_regs()+124: arch/arm/cpu/armv8/exceptions.S,
line 141
    4  0x7ff43740 efi_disk_read_blocks()+160: lib/efi_loader/efi_disk.c,

line 102

How and when did you get this stack trace?

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

When Abort happened I connected Xilinx debugger via jtag and look at cpu backtrace.

OK, but we are already in grub here and such a trace (in U-Boot) doesn't make sense. Right?

Correct grub already started. But I expect it is still using U-Boot drivers and all exception handlers are still in place from u-boot.

Yeah, but what I didn't understand was:

!"Synchronous Abort" handler, esr 0x02000000
!elr: ffffffffa816c5b0 lr : 000000000805e218 (reloc)
!elr: 00000000200005b0 lr : 000000007fef2218
(snip)
!Code: 000165fa 0b2d05de 0000ffff 00000000 (20000590)
!UEFI image [0x0000000077d48000:0x0000000077de5fff] '/efi\boot\bootaa64.efi'

Quoted 7 lines

"Code:" at the exception doesn't seem to be sane assembler, and "elr" is not within the code of neither U-Boot nor shim/grub(bootaa64.efi). ("esr" doesn't tell us anything.)
So I wondered where the backtrace came from.

BTW, can you please confirm which function sits at the address of "lr" (=0x7fe2218)?

I don't have that images anymore.

Quoted 4 lines
Quoted 3 lines

Maybe it is just sata/scsi related issue in EFI but weird is that when disks are scan just before command everything is working fine.

Quoted 8 lines

What do you mean by "when disks are scanned just before the command"? The case when you ran "run bootcmd_scsi" without "printenv -e"?

Do you reproduce the problem even if you revert the patch, "xilinx: zynqmp: Initialize usb and scsi via preboot", and run the commands, "run scsi_init; [printenv -e;] run bootcmd_scsi?

Can you also try other EFI commands, like "efidebug devices"?

I found that there is a difference if you run scsi reset or run scsi_init. When scsi_init is used I can't see any issue.

Variable looks like this
scsi_init=if ${scsi_need_init}; then scsi_need_init=false; scsi scan; fi

And when you run scsi scan (last log) you see that problem again. It means when scsi reset/scan is called twice issue is observed. In all cases efidebug devices are showing disk properly.

Thanks,
Michal

U-Boot SPL 2021.10-rc1-00278-gf17a954b24ad (Aug 17 2021 - 09:14:08 +0200)
PMUFW: v1.1
Loading new PMUFW cfg obj (2024 bytes)

Silicon version:        3
EL Level:       EL3
Chip ID:        zu9eg
Multiboot:      0
Trying to boot from MMC2
spl: could not initialize mmc. error: -19
Trying to boot from MMC1
spl_load_image_fat_os: error reading image u-boot.bin, err - -2
NOTICE:  BL31: Secure code at 0x7e000000
NOTICE:  BL31: Non secure code at 0x8000000
NOTICE:  BL31: v2.4(release):v2.4-592-gb5a757e853be
NOTICE:  BL31: Built : 07:08:43, Aug 12 2021

U-Boot 2021.10-rc1-00278-gf17a954b24ad (Aug 17 2021 - 09:14:08 +0200)

CPU:   ZynqMP
Silicon: v3
Model: ZynqMP ZCU102 Rev1.0
Board: Xilinx ZynqMP
DRAM:  4 GiB
PMUFW:  v1.1
Xilinx I2C Legacy format at nvmem0:
 Board name:    zcu102
 Board rev:     1.0
 Board SN:      847316301727-67998
 Ethernet mac:  00:0a:35:03:70:f6
EL Level:       EL2
Chip ID:        zu9eg
WDT:   Not found!
NAND:  0 MiB
MMC:   mmc@ff170000: 0

Loading Environment from FAT... *** Error - No Valid Environment Area found *** Warning - bad env area, using default environment

In:    serial
Out:   serial
Err:   serial
Bootmode: LVL_SHFT_SD_MODE1
Reset reason:   SRST

Net:
ZYNQ GEM: ff0e0000, mdio bus ff0e0000, phyaddr 12, interface rgmii-id
eth0: ethernet@ff0e0000
starting USB...
Bus dwc3@fe200000: Register 2000440 NbrPorts 2
Starting the controller
USB XHCI 1.00
scanning bus dwc3@fe200000 for devices... 1 USB Device(s) found
scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot: 0
ZynqMP>
ZynqMP>
ZynqMP> scsi reset

Reset SCSI
scanning bus for devices...
SATA link 0 timeout.
Target spinup took 0 ms.
AHCI 0001.0301 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq pm clo only pmp fbss pio slum part ccc apst
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
ZynqMP> efidebug devices
Scanning disk [email protected]...
Scanning disk ahci_scsi.id1lun0...
Found 5 disks
** Unable to read file ubootefi.var **
Failed to load EFI variables
Unable to find TPMv2 device
DFU alt info setting: done
Device Device Path
================ ====================
000000007be20810 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)
000000007be25800 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)
000000007be259d0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)/HD(1,0x01,0,0x2000,0x1cd2000) 000000007be25e30 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)
000000007be350d0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(1,GPT,85b731b6-a4b2-47f4-b1c6-aef6e0f2ce81,0x800,0xfffff) 000000007be35520 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(2,GPT,ac600dc7-3160-4f3c-a824-496d00e3d007,0x100800,0x18fff) 000000007be26210
/VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/MAC(000a350370f6,1)
ZynqMP> boot
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
BootOrder not defined
EFI boot manager: Cannot load any image
JTAG: Trying to boot script at 20000000
## Executing script at 20000000
Wrong image format for "source" command
JTAG: SCRIPT FAILED: continuing...
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
BootOrder not defined
EFI boot manager: Cannot load any image
MMC Device 1 not found
no mmc device at slot 1
SF: Detected n25q512a with page size 256 Bytes, erase size 64 KiB, total
64 MiB
device 0 offset 0x3e80000, size 0x80000
SF: 524288 bytes @ 0x3e80000 Read: OK
QSPI: Trying to boot script at 20000000
## Executing script at 20000000
Wrong image format for "source" command
QSPI: SCRIPT FAILED: continuing...

no devices available
NAND: SCRIPT FAILED: continuing...

Device 0: unknown device

Device 1: unknown device
scanning bus for devices...
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)

Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
... is now current device
Scanning scsi 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC BootOrder not defined
EFI boot manager: Cannot load any image
Found EFI removable media binary efi/boot/bootaa64.efi 647168 bytes read in 15 ms (41.1 MiB/s)
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Booting /efi\boot\bootaa64.efi
PE image measurement failed
Welcome to GRUB!

"Synchronous Abort" handler, esr 0x02000000
elr: ffffffffa816d690 lr : 000000000805f08c (reloc)
elr: 0000000020000690 lr : 000000007fef208c x0 : 0000000000000020 x1 : 0000000000000000 x2 : 00000000000003e0 x3 : 0000000077c88400 x4 : 0000000020000690 x5 : 000000007be178f0 x6 : 000000007ffa5038 x7 : 000000007ffa5038 x8 : 0000000000000006 x9 : fffffffffffffff0
x10: 000000007473696c x11: 000000007be178f0
x12: 00000000000003e0 x13: 0000000000000200
x14: 000000007fe94208 x15: 0000000000000000
x16: 000000007fecdcbc x17: 0000000000000000
x18: 000000007be12da0 x19: 000000007be253e0
x20: 0000000000000020 x21: 00000000000003e0
x22: 0000000000000000 x23: 0000000077c88400
x24: 0000000000000000 x25: 0000000000000100
x26: 000000000000000f x27: 00000000000001ff
x28: 000000007be4bed0 x29: 000000007be04920

Code: ffffffff ffffffff ffffffff ffffffff (ffffffff)

UEFI image [0x0000000077d46000:0x0000000077de3fff] '/efi\boot\bootaa64.efi' Resetting CPU ...

### ERROR ### Please RESET the board ###

U-Boot SPL 2021.10-rc1-00278-gf17a954b24ad (Aug 17 2021 - 09:14:08 +0200)
PMUFW: v1.1
Loading new PMUFW cfg obj (2024 bytes)

Silicon version:        3
EL Level:       EL3
Chip ID:        zu9eg
Multiboot:      0
Trying to boot from MMC2
spl: could not initialize mmc. error: -19
Trying to boot from MMC1
spl_load_image_fat_os: error reading image u-boot.bin, err - -2
NOTICE:  BL31: Secure code at 0x7e000000
NOTICE:  BL31: Non secure code at 0x8000000
NOTICE:  BL31: v2.4(release):v2.4-592-gb5a757e853be
NOTICE:  BL31: Built : 07:08:43, Aug 12 2021

U-Boot 2021.10-rc1-00278-gf17a954b24ad (Aug 17 2021 - 09:14:08 +0200)

CPU:   ZynqMP
Silicon: v3
Model: ZynqMP ZCU102 Rev1.0
Board: Xilinx ZynqMP
DRAM:  4 GiB
PMUFW:  v1.1
Xilinx I2C Legacy format at nvmem0:
 Board name:    zcu102
 Board rev:     1.0
 Board SN:      847316301727-67998
 Ethernet mac:  00:0a:35:03:70:f6
EL Level:       EL2
Chip ID:        zu9eg
WDT:   Not found!
NAND:  0 MiB
MMC:   mmc@ff170000: 0

Loading Environment from FAT... *** Error - No Valid Environment Area found *** Warning - bad env area, using default environment

In:    serial
Out:   serial
Err:   serial
Bootmode: LVL_SHFT_SD_MODE1
Reset reason:   SRST

Net:
ZYNQ GEM: ff0e0000, mdio bus ff0e0000, phyaddr 12, interface rgmii-id
eth0: ethernet@ff0e0000
starting USB...
Bus dwc3@fe200000: Register 2000440 NbrPorts 2
Starting the controller
USB XHCI 1.00
scanning bus dwc3@fe200000 for devices... 1 USB Device(s) found
scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot: 0
ZynqMP>
ZynqMP>
ZynqMP> run scsi_init
scanning bus for devices...
SATA link 0 timeout.
Target spinup took 0 ms.
AHCI 0001.0301 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq pm clo only pmp fbss pio slum part ccc apst
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
ZynqMP> efidebug devices
Scanning disk [email protected]...
Scanning disk ahci_scsi.id1lun0...
Found 5 disks
** Unable to read file ubootefi.var **
Failed to load EFI variables
Unable to find TPMv2 device
DFU alt info setting: done
Device Device Path
================ ====================
000000007be20790 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)
000000007be257e0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)
000000007be259b0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)/HD(1,0x01,0,0x2000,0x1cd2000) 000000007be25e10 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)
000000007be350d0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(1,GPT,85b731b6-a4b2-47f4-b1c6-aef6e0f2ce81,0x800,0xfffff) 000000007be354f0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(2,GPT,ac600dc7-3160-4f3c-a824-496d00e3d007,0x100800,0x18fff) 000000007be261f0
/VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/MAC(000a350370f6,1)
ZynqMP> boot
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
BootOrder not defined
EFI boot manager: Cannot load any image
JTAG: Trying to boot script at 20000000
## Executing script at 20000000
Wrong image format for "source" command
JTAG: SCRIPT FAILED: continuing...
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
BootOrder not defined
EFI boot manager: Cannot load any image
MMC Device 1 not found
no mmc device at slot 1
SF: Detected n25q512a with page size 256 Bytes, erase size 64 KiB, total
64 MiB
device 0 offset 0x3e80000, size 0x80000
SF: 524288 bytes @ 0x3e80000 Read: OK
QSPI: Trying to boot script at 20000000
## Executing script at 20000000
Wrong image format for "source" command
QSPI: SCRIPT FAILED: continuing...

no devices available
NAND: SCRIPT FAILED: continuing...

Device 0: unknown device

Device 1: unknown device

Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
... is now current device
Scanning scsi 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC BootOrder not defined
EFI boot manager: Cannot load any image
Found EFI removable media binary efi/boot/bootaa64.efi 647168 bytes read in 15 ms (41.1 MiB/s)
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Booting /efi\boot\bootaa64.efi
PE image measurement failed
Welcome to GRUB!

GNU GRUB version 2.11

 ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
 │*Linux BusyBox
                                           │

U-Boot SPL 2021.10-rc1-00278-gf17a954b24ad (Aug 17 2021 - 09:14:08 +0200)
PMUFW: v1.1
Loading new PMUFW cfg obj (2024 bytes)

Silicon version:        3
EL Level:       EL3
Chip ID:        zu9eg
Multiboot:      0
Trying to boot from MMC2
spl: could not initialize mmc. error: -19
Trying to boot from MMC1
spl_load_image_fat_os: error reading image u-boot.bin, err - -2
NOTICE:  BL31: Secure code at 0x7e000000
NOTICE:  BL31: Non secure code at 0x8000000
NOTICE:  BL31: v2.4(release):v2.4-592-gb5a757e853be
NOTICE:  BL31: Built : 07:08:43, Aug 12 2021

U-Boot 2021.10-rc1-00278-gf17a954b24ad (Aug 17 2021 - 09:14:08 +0200)

CPU:   ZynqMP
Silicon: v3
Model: ZynqMP ZCU102 Rev1.0
Board: Xilinx ZynqMP
DRAM:  4 GiB
PMUFW:  v1.1
Xilinx I2C Legacy format at nvmem0:
 Board name:    zcu102
 Board rev:     1.0
 Board SN:      847316301727-67998
 Ethernet mac:  00:0a:35:03:70:f6
EL Level:       EL2
Chip ID:        zu9eg
WDT:   Not found!
NAND:  0 MiB
MMC:   mmc@ff170000: 0

Loading Environment from FAT... *** Error - No Valid Environment Area found *** Warning - bad env area, using default environment

In:    serial
Out:   serial
Err:   serial
Bootmode: LVL_SHFT_SD_MODE1
Reset reason:   SRST

Net:
ZYNQ GEM: ff0e0000, mdio bus ff0e0000, phyaddr 12, interface rgmii-id
eth0: ethernet@ff0e0000
starting USB...
Bus dwc3@fe200000: Register 2000440 NbrPorts 2
Starting the controller
USB XHCI 1.00
scanning bus dwc3@fe200000 for devices... 1 USB Device(s) found
scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot: 0
ZynqMP>
ZynqMP>
ZynqMP> scsi scan
scanning bus for devices...
SATA link 0 timeout.
Target spinup took 0 ms.
AHCI 0001.0301 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq pm clo only pmp fbss pio slum part ccc apst
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
ZynqMP> efidebug devices
Scanning disk [email protected]...
Scanning disk ahci_scsi.id1lun0...
Found 5 disks
** Unable to read file ubootefi.var **
Failed to load EFI variables
Unable to find TPMv2 device
DFU alt info setting: done
Device Device Path
================ ====================
000000007be20810 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)
000000007be25800 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)
000000007be259d0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)/HD(1,0x01,0,0x2000,0x1cd2000) 000000007be25e30 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)
000000007be350d0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(1,GPT,85b731b6-a4b2-47f4-b1c6-aef6e0f2ce81,0x800,0xfffff) 000000007be35520 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(2,GPT,ac600dc7-3160-4f3c-a824-496d00e3d007,0x100800,0x18fff) 000000007be26210
/VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/MAC(000a350370f6,1)
ZynqMP> boot
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
BootOrder not defined
EFI boot manager: Cannot load any image
JTAG: Trying to boot script at 20000000
## Executing script at 20000000
Wrong image format for "source" command
JTAG: SCRIPT FAILED: continuing...
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
BootOrder not defined
EFI boot manager: Cannot load any image
MMC Device 1 not found
no mmc device at slot 1
SF: Detected n25q512a with page size 256 Bytes, erase size 64 KiB, total
64 MiB
device 0 offset 0x3e80000, size 0x80000
SF: 524288 bytes @ 0x3e80000 Read: OK
QSPI: Trying to boot script at 20000000
## Executing script at 20000000
Wrong image format for "source" command
QSPI: SCRIPT FAILED: continuing...

no devices available
NAND: SCRIPT FAILED: continuing...

Device 0: unknown device

Device 1: unknown device
scanning bus for devices...
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)

Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
... is now current device
Scanning scsi 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC BootOrder not defined
EFI boot manager: Cannot load any image
Found EFI removable media binary efi/boot/bootaa64.efi 647168 bytes read in 14 ms (44.1 MiB/s)
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Booting /efi\boot\bootaa64.efi
PE image measurement failed
Welcome to GRUB!

"Synchronous Abort" handler, esr 0x02000000
elr: ffffffffa816d690 lr : 000000000805f08c (reloc)
elr: 0000000020000690 lr : 000000007fef208c x0 : 0000000000000020 x1 : 0000000000000000 x2 : 00000000000003e0 x3 : 0000000077c88400 x4 : 0000000020000690 x5 : 000000007be178f0 x6 : 000000007ffa5038 x7 : 000000007ffa5038 x8 : 0000000000000006 x9 : fffffffffffffff0
x10: 000000007473696c x11: 000000007be178f0
x12: 00000000000003e0 x13: 0000000000000200
x14: 000000007fe94208 x15: 0000000000000000
x16: 000000007fecdcbc x17: 0000000000000000
x18: 000000007be12da0 x19: 000000007be253e0
x20: 0000000000000020 x21: 00000000000003e0
x22: 0000000000000000 x23: 0000000077c88400
x24: 0000000000000000 x25: 0000000000000100
x26: 000000000000000f x27: 00000000000001ff
x28: 000000007be4bed0 x29: 000000007be04920

Code: ffffffff ffffffff ffffffff ffffffff (ffffffff)

UEFI image [0x0000000077d46000:0x0000000077de3fff] '/efi\boot\bootaa64.efi' Resetting CPU ...

### ERROR ### Please RESET the board ###

#15 lore raw
Quoted 212 lines · Michal Simek
Quoted 192 lines · AKASHI Takahiro
Quoted 173 lines · Michal Simek
Quoted 170 lines · AKASHI Takahiro
Quoted 169 lines · Michal Simek
Quoted 165 lines · AKASHI Takahiro
Quoted 155 lines · Michal Simek

Hi,

Quoted 128 lines · AKASHI Takahiro
Quoted 120 lines · Michal Simek
Quoted 106 lines · Heinrich Schuchardt
Quoted 96 lines · Michal Simek

Hi,

Quoted 93 lines · Heinrich Schuchardt
Quoted 11 lines · Michal Simek

Hi,

I am playing with booting from USB via EFI. And I see very weird
behavior. I have burnt image with grub to USB flashdisk and I have
tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board.
On zcu102 grub is going to boot menu and everything is working fine as
expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list
partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

Quoted 5 lines

On zcu102(working board) I also see usb(gpt) partitions and SD.
grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

Quoted 3 lines

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This will not stop disk enumeration.

Quoted 42 lines

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic
xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they
are
the same. SOM doesn't have usb enabled by default (but I enabled it)
but
grub starts which means that communication with USB is fine.

It is based on my latest patches available here.
u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine.
ZynqMP> part list usb 0

Partition Map for USB device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name
          Attributes
          Type GUID
          Partition GUID
    1     0x00000800      0x001007fe      "Microsoft basic data"
          attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
          type:   data
          guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77
    2     0x00100800      0x001197fe      "Microsoft basic data"
          attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
          type:   data
          guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Do you have any idea why on one system is working fine to get to menu
and on others there is an issue to get all partitions even u-boot is
able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could be that the USB sub-system is simply not initialized yet when the boot manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough to run

     efidebug devices

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Quoted 8 lines
Quoted 2 lines

Block devices are enumerated in efi_disk_register(). Please, try to add debug output there to elucidate the problem.

I found where the problem is. First of all zcu102 didn't use the same image as others (it wasn't updated properly).
When you have CONFIG_EFI_CAPSULE_ON_DISK_EARLY that efi_disk_register() is called before usb block devices are detected and registered that's why grub doesn't see them.

Quoted 4 lines

The problem is CONFIG_EFI_SETUP_EARLY=y required by CONFIG_EFI_CAPSULE_ON_DISK_EARLY.

Why is USB initialized later then MMC?

It is not just usb. SCSI/sata are behaving in the same way too.

Quoted 4 lines

Overall we have a deficiency in the UEFI implementation in that we cannot deal with block devices added or removed after initialization.

Quoted 6 lines
Quoted 1 line

Here integration with the driver model is missing.

Right. And also there are commands which can create MBR partitions and I expect when you write image to SD and then run rescan or so you could get other partitions too.
Maybe hook via part_init()? with removing efi_disk_register.

Quoted 8 lines

For the record, I have proposed my ideas several times[1], [2]. I'm, however, no longer working on this issue as I have shifted my focus to UEFI secure boot and capsule update.

-Takahiro Akashi

[1] https://lists.denx.de/pipermail/u-boot/2018-November/347491.html
[2] https://lists.denx.de/pipermail/u-boot/2019-February/357923.html

I want to continue on this thread. I have disabled EFI_CAPSULE_ON_DISK_EARLY some time ago and trying to workaround that usb/scsi detection by simply calling usb reset and scsi reset as the part of PREBOOT. Then all disks are recorded and visible by grub.

But I found another issue which is kind of weird. We are using distroboot with soft of fixed sequence. Important part of sequence is sd, usb, scsi.

I have added grub on scsi and when I boot directly via run bootcmd_scsi0 everything is working fine. When I let distroboot to do the job it or run printenv -e before bootcmd_scsi0 I am getting exception. From debug it is visible that it is exception called from efi_disk_read_blocks.

Quoted 7 lines
    0  0x7ff5d188 hang()+20: include/bootstage.h, line 389
    1  0x7ff5f908 __assert_fail(): lib/panic.c, line 25
    2  0x7fe976a8 do_irq(): arch/arm/lib/interrupts_64.c, line 123
    3  0x7fe96a0c _restore_regs()+124: arch/arm/cpu/armv8/exceptions.S,
line 141
    4  0x7ff43740 efi_disk_read_blocks()+160: lib/efi_loader/efi_disk.c,

line 102

How and when did you get this stack trace?

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

When Abort happened I connected Xilinx debugger via jtag and look at cpu backtrace.

OK, but we are already in grub here and such a trace (in U-Boot) doesn't make sense. Right?

Correct grub already started. But I expect it is still using U-Boot drivers and all exception handlers are still in place from u-boot.

Yeah, but what I didn't understand was:

!"Synchronous Abort" handler, esr 0x02000000
!elr: ffffffffa816c5b0 lr : 000000000805e218 (reloc)
!elr: 00000000200005b0 lr : 000000007fef2218
(snip)
!Code: 000165fa 0b2d05de 0000ffff 00000000 (20000590)
!UEFI image [0x0000000077d48000:0x0000000077de5fff] '/efi\boot\bootaa64.efi'

Quoted 7 lines

"Code:" at the exception doesn't seem to be sane assembler, and "elr" is not within the code of neither U-Boot nor shim/grub(bootaa64.efi). ("esr" doesn't tell us anything.)
So I wondered where the backtrace came from.

BTW, can you please confirm which function sits at the address of "lr" (=0x7fe2218)?

I don't have that images anymore.

Quoted 7 lines
Quoted 3 lines

Maybe it is just sata/scsi related issue in EFI but weird is that when disks are scan just before command everything is working fine.

What do you mean by "when disks are scanned just before the command"? The case when you ran "run bootcmd_scsi" without "printenv -e"?

Quoted 8 lines
Quoted 5 lines

Do you reproduce the problem even if you revert the patch, "xilinx: zynqmp: Initialize usb and scsi via preboot", and run the commands, "run scsi_init; [printenv -e;] run bootcmd_scsi?

Can you also try other EFI commands, like "efidebug devices"?

I found that there is a difference if you run scsi reset or run scsi_init. When scsi_init is used I can't see any issue.

Here you have tried three cases:
(1) scsi reset; efidebug devices; boot (hence distro_bootcmd)
(2) run scsi_init; efidebug devices; boot
(3) scsi rescan; efidebug devices; boot

Only case(2) succeeded to boot the system. Right?

Please double-check that you don't see this problem in all those cases if you don't execute "efidebug devices" (or "printenv -e").

# make sure that no efi command will be executed before
# booting from scsi.
Quoted 5 lines

Variable looks like this
scsi_init=if ${scsi_need_init}; then scsi_need_init=false; scsi scan; fi

And when you run scsi scan (last log) you see that problem again. It means when scsi reset/scan is called twice issue is observed. In all

If this is true, my guess is:

* In the scenarios above, all the block devices are enumerated by scsi_scan() in the first "run reset" or "run rescan" and
new blk_desc's are created.
* efidebug is expected to execute efi_init_obj_list().
Please note:
EFI subsystem uses U-Boot's blk_desc internally to access block devices. Mapping between U-Boot's blk_desc and UEFI's efi_disk_obj (aka handle) is created only once and statically at the initialization in efi_init_obj_list().

* Now that scsi_scan() is executed again in the scond scsi command, all the block devices, hence blk_desc structures, will be freed by blk_unbind_all() and blk_desc's will be *re-created* by scsi probing.
* Nevertheless, the binding between blk_desc and efi_disk_obj is maintained even at this point, so any succeeding r/w operations via UEFI interfaces can point to bogus data of old blk_desc and therefore block accesses will get corrupted.

My guess above seems to be likely, but it doesn't explain well that loading/starting "grub" binary succeeds any way.

-Takahiro Akashi

Quoted 482 lines

cases efidebug devices are showing disk properly.

Thanks,
Michal

U-Boot SPL 2021.10-rc1-00278-gf17a954b24ad (Aug 17 2021 - 09:14:08 +0200)
PMUFW: v1.1
Loading new PMUFW cfg obj (2024 bytes)

Silicon version:        3
EL Level:       EL3
Chip ID:        zu9eg
Multiboot:      0
Trying to boot from MMC2
spl: could not initialize mmc. error: -19
Trying to boot from MMC1
spl_load_image_fat_os: error reading image u-boot.bin, err - -2
NOTICE:  BL31: Secure code at 0x7e000000
NOTICE:  BL31: Non secure code at 0x8000000
NOTICE:  BL31: v2.4(release):v2.4-592-gb5a757e853be
NOTICE:  BL31: Built : 07:08:43, Aug 12 2021

U-Boot 2021.10-rc1-00278-gf17a954b24ad (Aug 17 2021 - 09:14:08 +0200)

CPU:   ZynqMP
Silicon: v3
Model: ZynqMP ZCU102 Rev1.0
Board: Xilinx ZynqMP
DRAM:  4 GiB
PMUFW:  v1.1
Xilinx I2C Legacy format at nvmem0:
 Board name:    zcu102
 Board rev:     1.0
 Board SN:      847316301727-67998
 Ethernet mac:  00:0a:35:03:70:f6
EL Level:       EL2
Chip ID:        zu9eg
WDT:   Not found!
NAND:  0 MiB
MMC:   mmc@ff170000: 0

Loading Environment from FAT... *** Error - No Valid Environment Area found *** Warning - bad env area, using default environment

In:    serial
Out:   serial
Err:   serial
Bootmode: LVL_SHFT_SD_MODE1
Reset reason:   SRST

Net:
ZYNQ GEM: ff0e0000, mdio bus ff0e0000, phyaddr 12, interface rgmii-id
eth0: ethernet@ff0e0000
starting USB...
Bus dwc3@fe200000: Register 2000440 NbrPorts 2
Starting the controller
USB XHCI 1.00
scanning bus dwc3@fe200000 for devices... 1 USB Device(s) found
scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot: 0
ZynqMP>
ZynqMP>
ZynqMP> scsi reset

Reset SCSI
scanning bus for devices...
SATA link 0 timeout.
Target spinup took 0 ms.
AHCI 0001.0301 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq pm clo only pmp fbss pio slum part ccc apst
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
ZynqMP> efidebug devices
Scanning disk [email protected]...
Scanning disk ahci_scsi.id1lun0...
Found 5 disks
** Unable to read file ubootefi.var **
Failed to load EFI variables
Unable to find TPMv2 device
DFU alt info setting: done
Device Device Path
================ ====================
000000007be20810 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)
000000007be25800 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)
000000007be259d0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)/HD(1,0x01,0,0x2000,0x1cd2000) 000000007be25e30 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)
000000007be350d0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(1,GPT,85b731b6-a4b2-47f4-b1c6-aef6e0f2ce81,0x800,0xfffff) 000000007be35520 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(2,GPT,ac600dc7-3160-4f3c-a824-496d00e3d007,0x100800,0x18fff) 000000007be26210
/VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/MAC(000a350370f6,1)
ZynqMP> boot
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
BootOrder not defined
EFI boot manager: Cannot load any image
JTAG: Trying to boot script at 20000000
## Executing script at 20000000
Wrong image format for "source" command
JTAG: SCRIPT FAILED: continuing...
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
BootOrder not defined
EFI boot manager: Cannot load any image
MMC Device 1 not found
no mmc device at slot 1
SF: Detected n25q512a with page size 256 Bytes, erase size 64 KiB, total
64 MiB
device 0 offset 0x3e80000, size 0x80000
SF: 524288 bytes @ 0x3e80000 Read: OK
QSPI: Trying to boot script at 20000000
## Executing script at 20000000
Wrong image format for "source" command
QSPI: SCRIPT FAILED: continuing...

no devices available
NAND: SCRIPT FAILED: continuing...

Device 0: unknown device

Device 1: unknown device
scanning bus for devices...
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)

Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
... is now current device
Scanning scsi 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC BootOrder not defined
EFI boot manager: Cannot load any image
Found EFI removable media binary efi/boot/bootaa64.efi 647168 bytes read in 15 ms (41.1 MiB/s)
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Booting /efi\boot\bootaa64.efi
PE image measurement failed
Welcome to GRUB!

"Synchronous Abort" handler, esr 0x02000000
elr: ffffffffa816d690 lr : 000000000805f08c (reloc)
elr: 0000000020000690 lr : 000000007fef208c x0 : 0000000000000020 x1 : 0000000000000000 x2 : 00000000000003e0 x3 : 0000000077c88400 x4 : 0000000020000690 x5 : 000000007be178f0 x6 : 000000007ffa5038 x7 : 000000007ffa5038 x8 : 0000000000000006 x9 : fffffffffffffff0
x10: 000000007473696c x11: 000000007be178f0
x12: 00000000000003e0 x13: 0000000000000200
x14: 000000007fe94208 x15: 0000000000000000
x16: 000000007fecdcbc x17: 0000000000000000
x18: 000000007be12da0 x19: 000000007be253e0
x20: 0000000000000020 x21: 00000000000003e0
x22: 0000000000000000 x23: 0000000077c88400
x24: 0000000000000000 x25: 0000000000000100
x26: 000000000000000f x27: 00000000000001ff
x28: 000000007be4bed0 x29: 000000007be04920

Code: ffffffff ffffffff ffffffff ffffffff (ffffffff)

UEFI image [0x0000000077d46000:0x0000000077de3fff] '/efi\boot\bootaa64.efi' Resetting CPU ...

### ERROR ### Please RESET the board ###

U-Boot SPL 2021.10-rc1-00278-gf17a954b24ad (Aug 17 2021 - 09:14:08 +0200)
PMUFW: v1.1
Loading new PMUFW cfg obj (2024 bytes)

Silicon version:        3
EL Level:       EL3
Chip ID:        zu9eg
Multiboot:      0
Trying to boot from MMC2
spl: could not initialize mmc. error: -19
Trying to boot from MMC1
spl_load_image_fat_os: error reading image u-boot.bin, err - -2
NOTICE:  BL31: Secure code at 0x7e000000
NOTICE:  BL31: Non secure code at 0x8000000
NOTICE:  BL31: v2.4(release):v2.4-592-gb5a757e853be
NOTICE:  BL31: Built : 07:08:43, Aug 12 2021

U-Boot 2021.10-rc1-00278-gf17a954b24ad (Aug 17 2021 - 09:14:08 +0200)

CPU:   ZynqMP
Silicon: v3
Model: ZynqMP ZCU102 Rev1.0
Board: Xilinx ZynqMP
DRAM:  4 GiB
PMUFW:  v1.1
Xilinx I2C Legacy format at nvmem0:
 Board name:    zcu102
 Board rev:     1.0
 Board SN:      847316301727-67998
 Ethernet mac:  00:0a:35:03:70:f6
EL Level:       EL2
Chip ID:        zu9eg
WDT:   Not found!
NAND:  0 MiB
MMC:   mmc@ff170000: 0

Loading Environment from FAT... *** Error - No Valid Environment Area found *** Warning - bad env area, using default environment

In:    serial
Out:   serial
Err:   serial
Bootmode: LVL_SHFT_SD_MODE1
Reset reason:   SRST

Net:
ZYNQ GEM: ff0e0000, mdio bus ff0e0000, phyaddr 12, interface rgmii-id
eth0: ethernet@ff0e0000
starting USB...
Bus dwc3@fe200000: Register 2000440 NbrPorts 2
Starting the controller
USB XHCI 1.00
scanning bus dwc3@fe200000 for devices... 1 USB Device(s) found
scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot: 0
ZynqMP>
ZynqMP>
ZynqMP> run scsi_init
scanning bus for devices...
SATA link 0 timeout.
Target spinup took 0 ms.
AHCI 0001.0301 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq pm clo only pmp fbss pio slum part ccc apst
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
ZynqMP> efidebug devices
Scanning disk [email protected]...
Scanning disk ahci_scsi.id1lun0...
Found 5 disks
** Unable to read file ubootefi.var **
Failed to load EFI variables
Unable to find TPMv2 device
DFU alt info setting: done
Device Device Path
================ ====================
000000007be20790 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)
000000007be257e0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)
000000007be259b0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)/HD(1,0x01,0,0x2000,0x1cd2000) 000000007be25e10 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)
000000007be350d0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(1,GPT,85b731b6-a4b2-47f4-b1c6-aef6e0f2ce81,0x800,0xfffff) 000000007be354f0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(2,GPT,ac600dc7-3160-4f3c-a824-496d00e3d007,0x100800,0x18fff) 000000007be261f0
/VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/MAC(000a350370f6,1)
ZynqMP> boot
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
BootOrder not defined
EFI boot manager: Cannot load any image
JTAG: Trying to boot script at 20000000
## Executing script at 20000000
Wrong image format for "source" command
JTAG: SCRIPT FAILED: continuing...
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
BootOrder not defined
EFI boot manager: Cannot load any image
MMC Device 1 not found
no mmc device at slot 1
SF: Detected n25q512a with page size 256 Bytes, erase size 64 KiB, total
64 MiB
device 0 offset 0x3e80000, size 0x80000
SF: 524288 bytes @ 0x3e80000 Read: OK
QSPI: Trying to boot script at 20000000
## Executing script at 20000000
Wrong image format for "source" command
QSPI: SCRIPT FAILED: continuing...

no devices available
NAND: SCRIPT FAILED: continuing...

Device 0: unknown device

Device 1: unknown device

Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
... is now current device
Scanning scsi 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC BootOrder not defined
EFI boot manager: Cannot load any image
Found EFI removable media binary efi/boot/bootaa64.efi 647168 bytes read in 15 ms (41.1 MiB/s)
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Booting /efi\boot\bootaa64.efi
PE image measurement failed
Welcome to GRUB!

GNU GRUB version 2.11

 ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
 │*Linux BusyBox
                                           │

U-Boot SPL 2021.10-rc1-00278-gf17a954b24ad (Aug 17 2021 - 09:14:08 +0200)
PMUFW: v1.1
Loading new PMUFW cfg obj (2024 bytes)

Silicon version:        3
EL Level:       EL3
Chip ID:        zu9eg
Multiboot:      0
Trying to boot from MMC2
spl: could not initialize mmc. error: -19
Trying to boot from MMC1
spl_load_image_fat_os: error reading image u-boot.bin, err - -2
NOTICE:  BL31: Secure code at 0x7e000000
NOTICE:  BL31: Non secure code at 0x8000000
NOTICE:  BL31: v2.4(release):v2.4-592-gb5a757e853be
NOTICE:  BL31: Built : 07:08:43, Aug 12 2021

U-Boot 2021.10-rc1-00278-gf17a954b24ad (Aug 17 2021 - 09:14:08 +0200)

CPU:   ZynqMP
Silicon: v3
Model: ZynqMP ZCU102 Rev1.0
Board: Xilinx ZynqMP
DRAM:  4 GiB
PMUFW:  v1.1
Xilinx I2C Legacy format at nvmem0:
 Board name:    zcu102
 Board rev:     1.0
 Board SN:      847316301727-67998
 Ethernet mac:  00:0a:35:03:70:f6
EL Level:       EL2
Chip ID:        zu9eg
WDT:   Not found!
NAND:  0 MiB
MMC:   mmc@ff170000: 0

Loading Environment from FAT... *** Error - No Valid Environment Area found *** Warning - bad env area, using default environment

In:    serial
Out:   serial
Err:   serial
Bootmode: LVL_SHFT_SD_MODE1
Reset reason:   SRST

Net:
ZYNQ GEM: ff0e0000, mdio bus ff0e0000, phyaddr 12, interface rgmii-id
eth0: ethernet@ff0e0000
starting USB...
Bus dwc3@fe200000: Register 2000440 NbrPorts 2
Starting the controller
USB XHCI 1.00
scanning bus dwc3@fe200000 for devices... 1 USB Device(s) found
scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot: 0
ZynqMP>
ZynqMP>
ZynqMP> scsi scan
scanning bus for devices...
SATA link 0 timeout.
Target spinup took 0 ms.
AHCI 0001.0301 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq pm clo only pmp fbss pio slum part ccc apst
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
ZynqMP> efidebug devices
Scanning disk [email protected]...
Scanning disk ahci_scsi.id1lun0...
Found 5 disks
** Unable to read file ubootefi.var **
Failed to load EFI variables
Unable to find TPMv2 device
DFU alt info setting: done
Device Device Path
================ ====================
000000007be20810 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)
000000007be25800 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)
000000007be259d0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)/HD(1,0x01,0,0x2000,0x1cd2000) 000000007be25e30 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)
000000007be350d0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(1,GPT,85b731b6-a4b2-47f4-b1c6-aef6e0f2ce81,0x800,0xfffff) 000000007be35520 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(2,GPT,ac600dc7-3160-4f3c-a824-496d00e3d007,0x100800,0x18fff) 000000007be26210
/VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/MAC(000a350370f6,1)
ZynqMP> boot
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
BootOrder not defined
EFI boot manager: Cannot load any image
JTAG: Trying to boot script at 20000000
## Executing script at 20000000
Wrong image format for "source" command
JTAG: SCRIPT FAILED: continuing...
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
BootOrder not defined
EFI boot manager: Cannot load any image
MMC Device 1 not found
no mmc device at slot 1
SF: Detected n25q512a with page size 256 Bytes, erase size 64 KiB, total
64 MiB
device 0 offset 0x3e80000, size 0x80000
SF: 524288 bytes @ 0x3e80000 Read: OK
QSPI: Trying to boot script at 20000000
## Executing script at 20000000
Wrong image format for "source" command
QSPI: SCRIPT FAILED: continuing...

no devices available
NAND: SCRIPT FAILED: continuing...

Device 0: unknown device

Device 1: unknown device
scanning bus for devices...
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)

Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
... is now current device
Scanning scsi 0:1...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC BootOrder not defined
EFI boot manager: Cannot load any image
Found EFI removable media binary efi/boot/bootaa64.efi 647168 bytes read in 14 ms (44.1 MiB/s)
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Booting /efi\boot\bootaa64.efi
PE image measurement failed
Welcome to GRUB!

"Synchronous Abort" handler, esr 0x02000000
elr: ffffffffa816d690 lr : 000000000805f08c (reloc)
elr: 0000000020000690 lr : 000000007fef208c x0 : 0000000000000020 x1 : 0000000000000000 x2 : 00000000000003e0 x3 : 0000000077c88400 x4 : 0000000020000690 x5 : 000000007be178f0 x6 : 000000007ffa5038 x7 : 000000007ffa5038 x8 : 0000000000000006 x9 : fffffffffffffff0
x10: 000000007473696c x11: 000000007be178f0
x12: 00000000000003e0 x13: 0000000000000200
x14: 000000007fe94208 x15: 0000000000000000
x16: 000000007fecdcbc x17: 0000000000000000
x18: 000000007be12da0 x19: 000000007be253e0
x20: 0000000000000020 x21: 00000000000003e0
x22: 0000000000000000 x23: 0000000077c88400
x24: 0000000000000000 x25: 0000000000000100
x26: 000000000000000f x27: 00000000000001ff
x28: 000000007be4bed0 x29: 000000007be04920

Code: ffffffff ffffffff ffffffff ffffffff (ffffffff)

UEFI image [0x0000000077d46000:0x0000000077de3fff] '/efi\boot\bootaa64.efi' Resetting CPU ...

### ERROR ### Please RESET the board ###

#16 lore raw
Quoted 227 lines · AKASHI Takahiro
Quoted 212 lines · Michal Simek
Quoted 192 lines · AKASHI Takahiro
Quoted 173 lines · Michal Simek
Quoted 170 lines · AKASHI Takahiro
Quoted 169 lines · Michal Simek
Quoted 165 lines · AKASHI Takahiro
Quoted 155 lines · Michal Simek

Hi,

Quoted 128 lines · AKASHI Takahiro
Quoted 120 lines · Michal Simek
Quoted 106 lines · Heinrich Schuchardt
Quoted 96 lines · Michal Simek

Hi,

Quoted 93 lines · Heinrich Schuchardt

On 6/10/21 10:44 AM, Michal Simek wrote:
Hi,

I am playing with booting from USB via EFI. And I see very weird
behavior. I have burnt image with grub to USB flashdisk and I have
tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board.
On zcu102 grub is going to boot menu and everything is working fine as
expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list
partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

On zcu102(working board) I also see usb(gpt) partitions and SD.
grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This
will not stop disk enumeration.

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic
xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they
are
the same. SOM doesn't have usb enabled by default (but I enabled it)
but
grub starts which means that communication with USB is fine.

It is based on my latest patches available here.
u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine.
ZynqMP> part list usb 0

Partition Map for USB device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name
          Attributes
          Type GUID
          Partition GUID
    1     0x00000800      0x001007fe      "Microsoft basic data"
          attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
          type:   data
          guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77
    2     0x00100800      0x001197fe      "Microsoft basic data"
          attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
          type:   data
          guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Do you have any idea why on one system is working fine to get to menu
and on others there is an issue to get all partitions even u-boot is
able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could
be that the USB sub-system is simply not initialized yet when the boot
manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough
to run

     efidebug devices

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Quoted 8 lines
Quoted 2 lines

Block devices are enumerated in efi_disk_register(). Please, try to add
debug output there to elucidate the problem.

I found where the problem is. First of all zcu102 didn't use the same image as others (it wasn't updated properly).
When you have CONFIG_EFI_CAPSULE_ON_DISK_EARLY that efi_disk_register() is called before usb block devices are detected and registered that's why grub doesn't see them.

Quoted 4 lines

The problem is CONFIG_EFI_SETUP_EARLY=y required by CONFIG_EFI_CAPSULE_ON_DISK_EARLY.

Why is USB initialized later then MMC?

It is not just usb. SCSI/sata are behaving in the same way too.

Quoted 4 lines

Overall we have a deficiency in the UEFI implementation in that we cannot deal with block devices added or removed after initialization.

Quoted 6 lines
Quoted 1 line

Here integration with the driver model is missing.

Right. And also there are commands which can create MBR partitions and I expect when you write image to SD and then run rescan or so you could get other partitions too.
Maybe hook via part_init()? with removing efi_disk_register.

Quoted 8 lines

For the record, I have proposed my ideas several times[1], [2]. I'm, however, no longer working on this issue as I have shifted my focus to UEFI secure boot and capsule update.

-Takahiro Akashi

[1] https://lists.denx.de/pipermail/u-boot/2018-November/347491.html
[2] https://lists.denx.de/pipermail/u-boot/2019-February/357923.html

I want to continue on this thread. I have disabled EFI_CAPSULE_ON_DISK_EARLY some time ago and trying to workaround that usb/scsi detection by simply calling usb reset and scsi reset as the part of PREBOOT. Then all disks are recorded and visible by grub.

But I found another issue which is kind of weird. We are using distroboot with soft of fixed sequence. Important part of sequence is sd, usb, scsi.

I have added grub on scsi and when I boot directly via run bootcmd_scsi0 everything is working fine. When I let distroboot to do the job it or run printenv -e before bootcmd_scsi0 I am getting exception. From debug it is visible that it is exception called from efi_disk_read_blocks.

Quoted 7 lines
    0  0x7ff5d188 hang()+20: include/bootstage.h, line 389
    1  0x7ff5f908 __assert_fail(): lib/panic.c, line 25
    2  0x7fe976a8 do_irq(): arch/arm/lib/interrupts_64.c, line 123
    3  0x7fe96a0c _restore_regs()+124: arch/arm/cpu/armv8/exceptions.S,
line 141
    4  0x7ff43740 efi_disk_read_blocks()+160: lib/efi_loader/efi_disk.c,

line 102

How and when did you get this stack trace?

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

When Abort happened I connected Xilinx debugger via jtag and look at cpu backtrace.

OK, but we are already in grub here and such a trace (in U-Boot) doesn't make sense. Right?

Correct grub already started. But I expect it is still using U-Boot drivers and all exception handlers are still in place from u-boot.

Yeah, but what I didn't understand was:

!"Synchronous Abort" handler, esr 0x02000000
!elr: ffffffffa816c5b0 lr : 000000000805e218 (reloc)
!elr: 00000000200005b0 lr : 000000007fef2218
(snip)
!Code: 000165fa 0b2d05de 0000ffff 00000000 (20000590)
!UEFI image [0x0000000077d48000:0x0000000077de5fff] '/efi\boot\bootaa64.efi'

Quoted 7 lines

"Code:" at the exception doesn't seem to be sane assembler, and "elr" is not within the code of neither U-Boot nor shim/grub(bootaa64.efi). ("esr" doesn't tell us anything.)
So I wondered where the backtrace came from.

BTW, can you please confirm which function sits at the address of "lr" (=0x7fe2218)?

I don't have that images anymore.

Quoted 7 lines
Quoted 3 lines

Maybe it is just sata/scsi related issue in EFI but weird is that when disks are scan just before command everything is working fine.

What do you mean by "when disks are scanned just before the command"? The case when you ran "run bootcmd_scsi" without "printenv -e"?

Quoted 8 lines
Quoted 5 lines

Do you reproduce the problem even if you revert the patch, "xilinx: zynqmp: Initialize usb and scsi via preboot", and run the commands, "run scsi_init; [printenv -e;] run bootcmd_scsi?

Can you also try other EFI commands, like "efidebug devices"?

I found that there is a difference if you run scsi reset or run scsi_init. When scsi_init is used I can't see any issue.

Here you have tried three cases:
(1) scsi reset; efidebug devices; boot (hence distro_bootcmd)
(2) run scsi_init; efidebug devices; boot
(3) scsi rescan; efidebug devices; boot

Quoted 7 lines

Only case(2) succeeded to boot the system. Right?

Please double-check that you don't see this problem in all those cases if you don't execute "efidebug devices" (or "printenv -e").

# make sure that no efi command will be executed before
# booting from scsi.

I tested these 3 cases and all of them works fine.

scsi reset
devtype=scsi
run scan_dev_for_boot_part

run scsi_init
devtype=scsi
run scan_dev_for_boot_part

scsi rescan
devtype=scsi
run scan_dev_for_boot_part

Quoted 27 lines
Quoted 6 lines

Variable looks like this
scsi_init=if ${scsi_need_init}; then scsi_need_init=false; scsi scan; fi

And when you run scsi scan (last log) you see that problem again. It means when scsi reset/scan is called twice issue is observed. In all

If this is true, my guess is:

* In the scenarios above, all the block devices are enumerated by scsi_scan() in the first "run reset" or "run rescan" and
new blk_desc's are created.
* efidebug is expected to execute efi_init_obj_list().
Please note:
EFI subsystem uses U-Boot's blk_desc internally to access block devices. Mapping between U-Boot's blk_desc and UEFI's efi_disk_obj (aka handle) is created only once and statically at the initialization in efi_init_obj_list().

* Now that scsi_scan() is executed again in the scond scsi command, all the block devices, hence blk_desc structures, will be freed by blk_unbind_all() and blk_desc's will be *re-created* by scsi probing.
* Nevertheless, the binding between blk_desc and efi_disk_obj is maintained even at this point, so any succeeding r/w operations via UEFI interfaces can point to bogus data of old blk_desc and therefore block accesses will get corrupted.

Quoted 2 lines

My guess above seems to be likely, but it doesn't explain well that loading/starting "grub" binary succeeds any way.

That make sense what you described. I print desc and by reset there is new desc created at different address. And origin location is freed in device_unbind. Log is below.
The question is how to fix this behavior.

Thanks,
Michal

ZynqMP> scsi reset

Reset SCSI
scanning bus for devices...
blk_unbind_all: if_type 2
SATA link 0 timeout.
Target spinup took 0 ms.
AHCI 0001.0301 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq pm clo only pmp fbss pio slum part ccc apst
blk_create_device: devnum -1
blk_create_device: name ahci_scsi.id1lun0, desc 000000007be21340
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
ZynqMP> efidebug devices
Scanning disk [email protected]...
efi_disk_add_dev: desc 000000007be15b30
efi_disk_add_dev: desc 000000007be15b30
Scanning disk ahci_scsi.id1lun0...
efi_disk_add_dev: desc 000000007be21340
efi_disk_add_dev: desc 000000007be21340
efi_disk_add_dev: desc 000000007be21340
Found 5 disks
** Unable to read file ubootefi.var **
Failed to load EFI variables
Unable to find TPMv2 device
DFU alt info setting: done
Device Device Path
================ ====================
000000007be21590 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)
000000007be218a0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)
000000007be21a70 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)/HD(1,0x01,0,0x2000,0x1cd2000) 000000007be21f00 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)
000000007be222e0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(1,GPT,85b731b6-a4b2-47f4-b1c6-aef6e0f2ce81,0x800,0xfffff) 000000007be22730 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(2,GPT,ac600dc7-3160-4f3c-a824-496d00e3d007,0x100800,0x18fff) 000000007be22c80
/VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/MAC(000a350370f6,1)
ZynqMP> scsi reset

Reset SCSI
scanning bus for devices...
blk_unbind_all: if_type 2
Removing/unbinding device ahci_scsi.id1lun0
device_unbind: free desc 000000007be21340
blk_create_device: devnum -1
blk_create_device: name ahci_scsi.id1lun0, desc 000000007be3e070
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
ZynqMP>

#17 lore raw
Quoted 232 lines · Michal Simek
Quoted 229 lines · AKASHI Takahiro
Quoted 212 lines · Michal Simek
Quoted 192 lines · AKASHI Takahiro
Quoted 173 lines · Michal Simek
Quoted 170 lines · AKASHI Takahiro
Quoted 169 lines · Michal Simek
Quoted 165 lines · AKASHI Takahiro
Quoted 155 lines · Michal Simek

Hi,

Quoted 128 lines · AKASHI Takahiro
Quoted 120 lines · Michal Simek
Quoted 106 lines · Heinrich Schuchardt
Quoted 104 lines · Michal Simek

Hi,

On 6/10/21 11:47 AM, Heinrich Schuchardt wrote:
On 6/10/21 10:44 AM, Michal Simek wrote:
Hi,

I am playing with booting from USB via EFI. And I see very weird
behavior. I have burnt image with grub to USB flashdisk and I have
tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board.
On zcu102 grub is going to boot menu and everything is working fine as
expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list
partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

On zcu102(working board) I also see usb(gpt) partitions and SD.
grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This
will not stop disk enumeration.

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic
xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they
are
the same. SOM doesn't have usb enabled by default (but I enabled it)
but
grub starts which means that communication with USB is fine.

It is based on my latest patches available here.
u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine.
ZynqMP> part list usb 0

Partition Map for USB device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name
          Attributes
          Type GUID
          Partition GUID
    1     0x00000800      0x001007fe      "Microsoft basic data"
          attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
          type:   data
          guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77
    2     0x00100800      0x001197fe      "Microsoft basic data"
          attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
          type:   data
          guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Do you have any idea why on one system is working fine to get to menu
and on others there is an issue to get all partitions even u-boot is
able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could
be that the USB sub-system is simply not initialized yet when the boot
manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough
to run

     efidebug devices

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Block devices are enumerated in efi_disk_register(). Please, try to add
debug output there to elucidate the problem.

I found where the problem is. First of all zcu102 didn't use the same
image as others (it wasn't updated properly).
When you have CONFIG_EFI_CAPSULE_ON_DISK_EARLY that efi_disk_register()
is called before usb block devices are detected and registered that's
why grub doesn't see them.

Quoted 4 lines

The problem is CONFIG_EFI_SETUP_EARLY=y required by CONFIG_EFI_CAPSULE_ON_DISK_EARLY.

Why is USB initialized later then MMC?

It is not just usb. SCSI/sata are behaving in the same way too.

Quoted 4 lines

Overall we have a deficiency in the UEFI implementation in that we cannot deal with block devices added or removed after initialization.

Quoted 6 lines
Quoted 1 line

Here integration with the driver model is missing.

Right. And also there are commands which can create MBR partitions and I expect when you write image to SD and then run rescan or so you could get other partitions too.
Maybe hook via part_init()? with removing efi_disk_register.

Quoted 8 lines

For the record, I have proposed my ideas several times[1], [2]. I'm, however, no longer working on this issue as I have shifted my focus to UEFI secure boot and capsule update.

-Takahiro Akashi

[1] https://lists.denx.de/pipermail/u-boot/2018-November/347491.html
[2] https://lists.denx.de/pipermail/u-boot/2019-February/357923.html

I want to continue on this thread. I have disabled EFI_CAPSULE_ON_DISK_EARLY some time ago and trying to workaround that usb/scsi detection by simply calling usb reset and scsi reset as the part of PREBOOT. Then all disks are recorded and visible by grub.

But I found another issue which is kind of weird. We are using distroboot with soft of fixed sequence. Important part of sequence is sd, usb, scsi.

I have added grub on scsi and when I boot directly via run bootcmd_scsi0 everything is working fine. When I let distroboot to do the job it or run printenv -e before bootcmd_scsi0 I am getting exception. From debug it is visible that it is exception called from efi_disk_read_blocks.

Quoted 7 lines
    0  0x7ff5d188 hang()+20: include/bootstage.h, line 389
    1  0x7ff5f908 __assert_fail(): lib/panic.c, line 25
    2  0x7fe976a8 do_irq(): arch/arm/lib/interrupts_64.c, line 123
    3  0x7fe96a0c _restore_regs()+124: arch/arm/cpu/armv8/exceptions.S,
line 141
    4  0x7ff43740 efi_disk_read_blocks()+160: lib/efi_loader/efi_disk.c,

line 102

How and when did you get this stack trace?

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

When Abort happened I connected Xilinx debugger via jtag and look at cpu backtrace.

OK, but we are already in grub here and such a trace (in U-Boot) doesn't make sense. Right?

Correct grub already started. But I expect it is still using U-Boot drivers and all exception handlers are still in place from u-boot.

Yeah, but what I didn't understand was:

!"Synchronous Abort" handler, esr 0x02000000
!elr: ffffffffa816c5b0 lr : 000000000805e218 (reloc)
!elr: 00000000200005b0 lr : 000000007fef2218
(snip)
!Code: 000165fa 0b2d05de 0000ffff 00000000 (20000590)
!UEFI image [0x0000000077d48000:0x0000000077de5fff] '/efi\boot\bootaa64.efi'

Quoted 7 lines

"Code:" at the exception doesn't seem to be sane assembler, and "elr" is not within the code of neither U-Boot nor shim/grub(bootaa64.efi). ("esr" doesn't tell us anything.)
So I wondered where the backtrace came from.

BTW, can you please confirm which function sits at the address of "lr" (=0x7fe2218)?

I don't have that images anymore.

Quoted 7 lines
Quoted 3 lines

Maybe it is just sata/scsi related issue in EFI but weird is that when disks are scan just before command everything is working fine.

What do you mean by "when disks are scanned just before the command"? The case when you ran "run bootcmd_scsi" without "printenv -e"?

Quoted 8 lines
Quoted 5 lines

Do you reproduce the problem even if you revert the patch, "xilinx: zynqmp: Initialize usb and scsi via preboot", and run the commands, "run scsi_init; [printenv -e;] run bootcmd_scsi?

Can you also try other EFI commands, like "efidebug devices"?

I found that there is a difference if you run scsi reset or run scsi_init. When scsi_init is used I can't see any issue.

Here you have tried three cases:
(1) scsi reset; efidebug devices; boot (hence distro_bootcmd)
(2) run scsi_init; efidebug devices; boot
(3) scsi rescan; efidebug devices; boot

Only case(2) succeeded to boot the system. Right?

Quoted 7 lines
Quoted 5 lines

Please double-check that you don't see this problem in all those cases if you don't execute "efidebug devices" (or "printenv -e").

# make sure that no efi command will be executed before
# booting from scsi.

I tested these 3 cases and all of them works fine.

Thank you for the confirmation.

Quoted 4 lines

scsi reset
devtype=scsi
run scan_dev_for_boot_part

Quoted 39 lines

run scsi_init
devtype=scsi
run scan_dev_for_boot_part

scsi rescan
devtype=scsi
run scan_dev_for_boot_part

Quoted 29 lines
Quoted 6 lines

Variable looks like this
scsi_init=if ${scsi_need_init}; then scsi_need_init=false; scsi scan; fi

And when you run scsi scan (last log) you see that problem again. It means when scsi reset/scan is called twice issue is observed. In all

If this is true, my guess is:

* In the scenarios above, all the block devices are enumerated by scsi_scan() in the first "run reset" or "run rescan" and
new blk_desc's are created.
* efidebug is expected to execute efi_init_obj_list().
Please note:
EFI subsystem uses U-Boot's blk_desc internally to access block devices. Mapping between U-Boot's blk_desc and UEFI's efi_disk_obj (aka handle) is created only once and statically at the initialization in efi_init_obj_list().

* Now that scsi_scan() is executed again in the scond scsi command, all the block devices, hence blk_desc structures, will be freed by blk_unbind_all() and blk_desc's will be *re-created* by scsi probing.
* Nevertheless, the binding between blk_desc and efi_disk_obj is maintained even at this point, so any succeeding r/w operations via UEFI interfaces can point to bogus data of old blk_desc and therefore block accesses will get corrupted.

My guess above seems to be likely, but it doesn't explain well that loading/starting "grub" binary succeeds any way.

# The implementation of LoadImage interface doesn't use block_io_protocol,

# and so we won't see this problem when 'grub' is started.
Quoted 4 lines

That make sense what you described. I print desc and by reset there is new desc created at different address. And origin location is freed in device_unbind. Log is below.
The question is how to fix this behavior.

It is a matter of *integration* of U-Boot's DM and UEFI implementation. It can be, however, a bit difficult/complicated task to achieve this goal in such a way that Simon has expected (for example, see [1]).

-Takahiro Akashi

[1] https://lists.denx.de/pipermail/u-boot/2021-June/452847.html

Quoted 56 lines

Thanks,
Michal

ZynqMP> scsi reset

Reset SCSI
scanning bus for devices...
blk_unbind_all: if_type 2
SATA link 0 timeout.
Target spinup took 0 ms.
AHCI 0001.0301 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq pm clo only pmp fbss pio slum part ccc apst
blk_create_device: devnum -1
blk_create_device: name ahci_scsi.id1lun0, desc 000000007be21340
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
ZynqMP> efidebug devices
Scanning disk [email protected]...
efi_disk_add_dev: desc 000000007be15b30
efi_disk_add_dev: desc 000000007be15b30
Scanning disk ahci_scsi.id1lun0...
efi_disk_add_dev: desc 000000007be21340
efi_disk_add_dev: desc 000000007be21340
efi_disk_add_dev: desc 000000007be21340
Found 5 disks
** Unable to read file ubootefi.var **
Failed to load EFI variables
Unable to find TPMv2 device
DFU alt info setting: done
Device Device Path
================ ====================
000000007be21590 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)
000000007be218a0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)
000000007be21a70 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)/HD(1,0x01,0,0x2000,0x1cd2000) 000000007be21f00 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)
000000007be222e0 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(1,GPT,85b731b6-a4b2-47f4-b1c6-aef6e0f2ce81,0x800,0xfffff) 000000007be22730 /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(1,0)/HD(2,GPT,ac600dc7-3160-4f3c-a824-496d00e3d007,0x100800,0x18fff) 000000007be22c80
/VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/MAC(000a350370f6,1)
ZynqMP> scsi reset

Reset SCSI
scanning bus for devices...
blk_unbind_all: if_type 2
Removing/unbinding device ahci_scsi.id1lun0
device_unbind: free desc 000000007be21340
blk_create_device: devnum -1
blk_create_device: name ahci_scsi.id1lun0, desc 000000007be3e070
Device 0: (1:0) Vendor: ATA Prod.: Maxtor 7V300F0 Rev: VA11
Type: Hard Disk
Capacity: 286188.8 MB = 279.4 GB (586114704 x 512)
ZynqMP>

#18 lore raw
Quoted 295 lines · AKASHI Takahiro
Quoted 232 lines · Michal Simek
Quoted 229 lines · AKASHI Takahiro
Quoted 212 lines · Michal Simek
Quoted 192 lines · AKASHI Takahiro
Quoted 173 lines · Michal Simek
Quoted 170 lines · AKASHI Takahiro
Quoted 169 lines · Michal Simek
Quoted 165 lines · AKASHI Takahiro
Quoted 155 lines · Michal Simek

Hi,

Quoted 128 lines · AKASHI Takahiro
Quoted 120 lines · Michal Simek
Quoted 110 lines · Heinrich Schuchardt

On 6/10/21 12:04 PM, Michal Simek wrote:
Hi,

On 6/10/21 11:47 AM, Heinrich Schuchardt wrote:
On 6/10/21 10:44 AM, Michal Simek wrote:
Hi,

I am playing with booting from USB via EFI. And I see very weird
behavior. I have burnt image with grub to USB flashdisk and I have
tested it on 3 zynqmp boards. zcu102, zcu104 and SOM Kria board.
On zcu102 grub is going to boot menu and everything is working fine as
expected.
On zcu104 and SOM Kria I am able to get grub not to menu. When I list
partitions in grub I see that only SDs are listed:
grub> ls
(hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Hello Michal,

thanks for sharing your observations.

What devices do hd0 and hd1 relate to?

On zcu102(working board) I also see usb(gpt) partitions and SD.
grub> ls
(hd0) (hd0,gpt2) (hd0,gpt1) (hd1) (hd1,msdos1)

GPT and MBR partitioning are independent of the device type.

On zcu104 I see one more error message
"PE image measurement failed"

This is related to CONFIG_EFI_TCG2_PROTOCOL=y. Do you have a TPMv2? This
will not stop disk enumeration.

But I can't see it on SOM.

U-Boot image is just the same for all boards. I am using generic
xilinx_zynqmp_virt_defconfig.

When I compare DT description for USB between zcu102 and zcu104 they
are
the same. SOM doesn't have usb enabled by default (but I enabled it)
but
grub starts which means that communication with USB is fine.

It is based on my latest patches available here.
u-boot/custodians/u-boot-microblaze.git (usb-efi-issue branch)

Also when I list usb I see all partitions just fine.
ZynqMP> part list usb 0

Partition Map for USB device 0  --   Partition Type: EFI

Part    Start LBA       End LBA         Name
          Attributes
          Type GUID
          Partition GUID
    1     0x00000800      0x001007fe      "Microsoft basic data"
          attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
          type:   data
          guid:   0e7f8b3d-296b-4720-be9d-c4687d3c4a77
    2     0x00100800      0x001197fe      "Microsoft basic data"
          attrs:  0x0000000000000000
          type:   ebd0a0a2-b9e5-4433-87c0-68b6b72699c7
          type:   data
          guid:   8892eddc-231a-4e6e-a5e1-c310f4482fb7

Do you have any idea why on one system is working fine to get to menu
and on others there is an issue to get all partitions even u-boot is
able to see them and can work with them.

Thanks,
Michal

Where is the GRUB binary? - If it is in EFI/boot/bootaa64.efi, it could
be that the USB sub-system is simply not initialized yet when the boot
manager is called by distroboot.

For testing partition detection in the UEFI sub-system it is enough
to run

     efidebug devices

Until yesterday we had a problem with partition numbers >= 10, cf.

efi_loader: partition numbers are hexadecimal
https://source.denx.de/u-boot/u-boot/-/commit/3dca77b1dc1b6dbf9c8b51572fe4b0553cef009f

Block devices are enumerated in efi_disk_register(). Please, try to add
debug output there to elucidate the problem.

I found where the problem is. First of all zcu102 didn't use the same
image as others (it wasn't updated properly).
When you have CONFIG_EFI_CAPSULE_ON_DISK_EARLY that efi_disk_register()
is called before usb block devices are detected and registered that's
why grub doesn't see them.

The problem is CONFIG_EFI_SETUP_EARLY=y required by
CONFIG_EFI_CAPSULE_ON_DISK_EARLY.

Why is USB initialized later then MMC?

It is not just usb. SCSI/sata are behaving in the same way too.

Quoted 4 lines

Overall we have a deficiency in the UEFI implementation in that we
cannot deal with block devices added or removed after initialization.

Quoted 6 lines
Quoted 1 line

Here integration with the driver model is missing.

Right. And also there are commands which can create MBR partitions and I expect when you write image to SD and then run rescan or so you could get other partitions too.
Maybe hook via part_init()? with removing efi_disk_register.

Quoted 8 lines

For the record, I have proposed my ideas several times[1], [2]. I'm, however, no longer working on this issue as I have shifted my focus to UEFI secure boot and capsule update.

-Takahiro Akashi

[1] https://lists.denx.de/pipermail/u-boot/2018-November/347491.html
[2] https://lists.denx.de/pipermail/u-boot/2019-February/357923.html

I want to continue on this thread. I have disabled EFI_CAPSULE_ON_DISK_EARLY some time ago and trying to workaround that usb/scsi detection by simply calling usb reset and scsi reset as the part of PREBOOT. Then all disks are recorded and visible by grub.

But I found another issue which is kind of weird. We are using distroboot with soft of fixed sequence. Important part of sequence is sd, usb, scsi.

I have added grub on scsi and when I boot directly via run bootcmd_scsi0 everything is working fine. When I let distroboot to do the job it or run printenv -e before bootcmd_scsi0 I am getting exception. From debug it is visible that it is exception called from efi_disk_read_blocks.

Quoted 7 lines
    0  0x7ff5d188 hang()+20: include/bootstage.h, line 389
    1  0x7ff5f908 __assert_fail(): lib/panic.c, line 25
    2  0x7fe976a8 do_irq(): arch/arm/lib/interrupts_64.c, line 123
    3  0x7fe96a0c _restore_regs()+124: arch/arm/cpu/armv8/exceptions.S,
line 141
    4  0x7ff43740 efi_disk_read_blocks()+160: lib/efi_loader/efi_disk.c,

line 102

How and when did you get this stack trace?

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

When Abort happened I connected Xilinx debugger via jtag and look at cpu backtrace.

OK, but we are already in grub here and such a trace (in U-Boot) doesn't make sense. Right?

Correct grub already started. But I expect it is still using U-Boot drivers and all exception handlers are still in place from u-boot.

Yeah, but what I didn't understand was:

!"Synchronous Abort" handler, esr 0x02000000
!elr: ffffffffa816c5b0 lr : 000000000805e218 (reloc)
!elr: 00000000200005b0 lr : 000000007fef2218
(snip)
!Code: 000165fa 0b2d05de 0000ffff 00000000 (20000590)
!UEFI image [0x0000000077d48000:0x0000000077de5fff] '/efi\boot\bootaa64.efi'

Quoted 7 lines

"Code:" at the exception doesn't seem to be sane assembler, and "elr" is not within the code of neither U-Boot nor shim/grub(bootaa64.efi). ("esr" doesn't tell us anything.)
So I wondered where the backtrace came from.

BTW, can you please confirm which function sits at the address of "lr" (=0x7fe2218)?

I don't have that images anymore.

Quoted 7 lines
Quoted 3 lines

Maybe it is just sata/scsi related issue in EFI but weird is that when disks are scan just before command everything is working fine.

What do you mean by "when disks are scanned just before the command"? The case when you ran "run bootcmd_scsi" without "printenv -e"?

Quoted 8 lines
Quoted 5 lines

Do you reproduce the problem even if you revert the patch, "xilinx: zynqmp: Initialize usb and scsi via preboot", and run the commands, "run scsi_init; [printenv -e;] run bootcmd_scsi?

Can you also try other EFI commands, like "efidebug devices"?

I found that there is a difference if you run scsi reset or run scsi_init. When scsi_init is used I can't see any issue.

Here you have tried three cases:
(1) scsi reset; efidebug devices; boot (hence distro_bootcmd)
(2) run scsi_init; efidebug devices; boot
(3) scsi rescan; efidebug devices; boot

Only case(2) succeeded to boot the system. Right?

Quoted 7 lines
Quoted 5 lines

Please double-check that you don't see this problem in all those cases if you don't execute "efidebug devices" (or "printenv -e").

# make sure that no efi command will be executed before
# booting from scsi.

I tested these 3 cases and all of them works fine.

Thank you for the confirmation.

Quoted 4 lines

scsi reset
devtype=scsi
run scan_dev_for_boot_part

Quoted 39 lines

run scsi_init
devtype=scsi
run scan_dev_for_boot_part

scsi rescan
devtype=scsi
run scan_dev_for_boot_part

Quoted 29 lines
Quoted 6 lines

Variable looks like this
scsi_init=if ${scsi_need_init}; then scsi_need_init=false; scsi scan; fi

And when you run scsi scan (last log) you see that problem again. It means when scsi reset/scan is called twice issue is observed. In all

If this is true, my guess is:

* In the scenarios above, all the block devices are enumerated by scsi_scan() in the first "run reset" or "run rescan" and
new blk_desc's are created.
* efidebug is expected to execute efi_init_obj_list().
Please note:
EFI subsystem uses U-Boot's blk_desc internally to access block devices. Mapping between U-Boot's blk_desc and UEFI's efi_disk_obj (aka handle) is created only once and statically at the initialization in efi_init_obj_list().

* Now that scsi_scan() is executed again in the scond scsi command, all the block devices, hence blk_desc structures, will be freed by blk_unbind_all() and blk_desc's will be *re-created* by scsi probing.
* Nevertheless, the binding between blk_desc and efi_disk_obj is maintained even at this point, so any succeeding r/w operations via UEFI interfaces can point to bogus data of old blk_desc and therefore block accesses will get corrupted.

My guess above seems to be likely, but it doesn't explain well that loading/starting "grub" binary succeeds any way.

# The implementation of LoadImage interface doesn't use block_io_protocol,

# and so we won't see this problem when 'grub' is started.
Quoted 4 lines

That make sense what you described. I print desc and by reset there is new desc created at different address. And origin location is freed in device_unbind. Log is below.
The question is how to fix this behavior.

Quoted 7 lines

It is a matter of *integration* of U-Boot's DM and UEFI implementation. It can be, however, a bit difficult/complicated task to achieve this goal in such a way that Simon has expected (for example, see [1]).

-Takahiro Akashi

[1] https://lists.denx.de/pipermail/u-boot/2021-June/452847.html

Ok. Then for me the only reasonable solution which is available now is to call scsi_init via preboot to get block device before any efi initialization.
And likely usb_boot should be updated in the same way as scsi to use variable and not call usb start/reset twice.

Thanks,
Michal