BUG: SiFive Unmatched: BAR0 on video card is never initialized
Hi Heinrich; hi all owners of SiFive Unmatched + PCIe video card,
Default U-Boot configuration for Unmatched always fails to initialize BAR0 of my video card (NVidia GT710), because the PCI driver selects the MEMORY region at 0x70000000 for bus_start/phys_start ("region 2", according to decode_regions() output), which has the size of 0x1000000. Naturally, the video card's BAR0 doesn't fit into this small region, because its size is also 0x1000000:
pci_auto_config_devices: start
PCI Autoconfig: Bus Memory region: [70000000-70ffffff],
Physical Memory [70000000-70ffffff]
PCI Autoconfig: Bus Prefetchable Mem region: [2000000000-3fffffffff],
Physical Memory [2000000000-3fffffffff]
PCI Autoconfig: Bus I/O region: [60080000-6008ffff],
Physical Memory [60080000-6008ffff] .......
pci_auto_config_devices: device pci_7:0.0
PCI Autoconfig: BAR 0, Mem, size=0x1000000, bus_lower=0x70400000, bus_start=0x70000000, phys_start=0x70000000 addr = 0x71000000
No room in resource, avail start=70400000 / size=1000000, need=1000000
PCI: Failed autoconfig bar 10
The situation changes drastically if CONFIG_PCI_REGION_MULTI_ENTRY is set to "yes". After this, PCI starts to use "region 1" for MEMORY, which has the size of 0xFF70000, and is, obviously, suitable for GPU's BAR0:
pci_uclass_pre_probe, bus=-1/pcie@e00000000, parent=socdecode_regions: len=28, cells_per_record=7decode_regions: region 0, pci_addr=60080000, addr=60080000, size=10000, space_code=1
- type=1, pos=0decode_regions: region 1, pci_addr=60090000, addr=60090000, size=ff70000, space_code=2
- type=0, pos=1decode_regions: region 2, pci_addr=70000000, addr=70000000, size=1000000, space_code=2
- type=0, pos=2decode_regions: region 3, pci_addr=2000000000, addr=2000000000, size=2000000000, space_code=3
- type=8, pos=3
....
PCIE-0: Link up (Gen1-x8, Bus0)pci_uclass_post_probe: probing bus 0pci_bind_bus_devices: bus 0/pcie@e00000000: found device 0, function 0: find ret=-19pci_find_and_bind_driver: Searching for driver: vendor=f15e, device=0pci_find_and_bind_driver: No match found: bound generic driver insteadpci_auto_config_devices: start
PCI Autoconfig: Bus Memory region: [60090000-6fffffff],
Physical Memory [60090000-6fffffff]
PCI Autoconfig: Bus Prefetchable Mem region: [2000000000-3fffffffff],
Physical Memory [2000000000-3fffffffff]
PCI Autoconfig: Bus I/O region: [60080000-6008ffff],
Physical Memory [60080000-6008ffff]This picture is in full accord with Linux kernel, which also uses 0x69000000 ("region 1") for bus_start and phys_start.
With this setting, all BARs on all devices are initialized correctly, and, moreover, I can access memory-mapped VGA ports of the card (see [1] for details), which is essential for initializing the card in text mode with bios_emulator:
PCI Autoconfig: BAR 0, Mem, size=0x1000000, bus_lower=0x60500000, bus_start=0x60090000, phys_start=0x60090000 address=0x61000000 bus_lower=0x62000000
PCI Autoconfig: BAR 1, Prf64, size=0x8000000, bus_lower=0x2000000000, bus_start=0x2000000000, phys_start=0x2000000000 address=0x2000000000 bus_lower=0x2008000000
PCI Autoconfig: BAR 2, Prf64, size=0x2000000, bus_lower=0x2008000000, bus_start=0x2000000000, phys_start=0x2000000000 address=0x2008000000 bus_lower=0x200a000000
PCI Autoconfig: BAR 3, I/O, size=0x80, bus_lower=0x60080000, bus_start=0x60080000, phys_start=0x60080000 address=0x60080000 bus_lower=0x60080080
PCI Autoconfig: ROM, size=0x80000, bus_lower=0x62000000, bus_start=0x60090000, phys_start=0x60090000 address=0x62000000 bus_lower=0x62080000
Notes:
[1]: https://envytools.readthedocs.io/en/latest/hw/mmio.html#gf100-mmio-map
Signature · 2 lines
Regards, Yuri
Am 19. November 2024 21:45:39 MEZ schrieb Yuri Zaporozhets <[email protected]>:
Quoted 71 lines
Hi Heinrich; hi all owners of SiFive Unmatched + PCIe video card,
Default U-Boot configuration for Unmatched always fails to initialize BAR0 of my video card (NVidia GT710), because the PCI driver selects the MEMORY region at
0x70000000for bus_start/phys_start ("region 2", according todecode_regions()output), which has the size of0x1000000. Naturally, the video card's BAR0 doesn't fit into this small region, because its size is also0x1000000:
pci_auto_config_devices: start
PCI Autoconfig: Bus Memory region: [70000000-70ffffff],Physical Memory [70000000-70ffffff] PCI Autoconfig: Bus Prefetchable Mem region: [2000000000-3fffffffff], Physical Memory [2000000000-3fffffffff] PCI Autoconfig: Bus I/O region: [60080000-6008ffff], Physical Memory [60080000-6008ffff].......
pci_auto_config_devices: device pci_7:0.0
PCI Autoconfig: BAR 0, Mem, size=0x1000000, bus_lower=0x70400000, bus_start=0x70000000, phys_start=0x70000000 addr =0x71000000
No room in resource, avail start=70400000 / size=1000000, need=1000000
PCI: Failed autoconfig bar 10The situation changes drastically if
CONFIG_PCI_REGION_MULTI_ENTRYis set to "yes". After this, PCI starts to use "region 1" for MEMORY, which has the size of0xFF70000, and is, obviously, suitable for GPU's BAR0:
pci_uclass_pre_probe, bus=-1/pcie@e00000000, parent=socdecode_regions: len=28, cells_per_record=7decode_regions: region 0, pci_addr=60080000, addr=60080000, size=10000, space_code=1
- type=1, pos=0decode_regions: region 1, pci_addr=60090000, addr=60090000, size=ff70000, space_code=2
- type=0, pos=1decode_regions: region 2, pci_addr=70000000, addr=70000000, size=1000000, space_code=2
- type=0, pos=2decode_regions: region 3, pci_addr=2000000000, addr=2000000000, size=2000000000, space_code=3
- type=8, pos=3
....
PCIE-0: Link up (Gen1-x8, Bus0)pci_uclass_post_probe: probing bus 0pci_bind_bus_devices: bus 0/pcie@e00000000: found device 0, function 0: find ret=-19pci_find_and_bind_driver: Searching for driver: vendor=f15e, device=0pci_find_and_bind_driver: No match found: bound generic driver insteadpci_auto_config_devices: start
PCI Autoconfig: Bus Memory region: [60090000-6fffffff],Physical Memory [60090000-6fffffff] PCI Autoconfig: Bus Prefetchable Mem region: [2000000000-3fffffffff], Physical Memory [2000000000-3fffffffff] PCI Autoconfig: Bus I/O region: [60080000-6008ffff], Physical Memory [60080000-6008ffff]This picture is in full accord with Linux kernel, which also uses
0x69000000("region 1") forbus_startandphys_start.With this setting, all BARs on all devices are initialized correctly, and, moreover, I can access memory-mapped VGA ports of the card (see [1] for details), which is essential for initializing the card in text mode with
bios_emulator:PCI Autoconfig: BAR 0, Mem, size=0x1000000, bus_lower=0x60500000, bus_start=0x60090000, phys_start=0x60090000 address=0x61000000 bus_lower=0x62000000
PCI Autoconfig: BAR 1, Prf64, size=0x8000000, bus_lower=0x2000000000, bus_start=0x2000000000, phys_start=0x2000000000 address=0x2000000000 bus_lower=0x2008000000
PCI Autoconfig: BAR 2, Prf64, size=0x2000000, bus_lower=0x2008000000, bus_start=0x2000000000, phys_start=0x2000000000 address=0x2008000000 bus_lower=0x200a000000
PCI Autoconfig: BAR 3, I/O, size=0x80, bus_lower=0x60080000, bus_start=0x60080000, phys_start=0x60080000 address=0x60080000 bus_lower=0x60080080
Quoted 7 lines
PCI Autoconfig: ROM, size=0x80000, bus_lower=0x62000000, bus_start=0x60090000, phys_start=0x60090000 address=0x62000000 bus_lower=0x62080000
Notes:
[1]: https://envytools.readthedocs.io/en/latest/hw/mmio.html#gf100-mmio-map
Hello Yuri,
Thank you for sharing your investigation results. Could you, please, send a patch?
Were you able to show output on HDMI?
Best regards
Heinrich
Hi Heinrich,
Quoted 4 lines · Heinrich Schuchardt
Hello Yuri,
Thank you for sharing your investigation results. Could you, please, send a patch?
Sure. Already did.
Quoted 1 line
Were you able to show output on HDMI?
Oh, not that fast -- I'm working on it :-) . Currently, I have my 1992 Hitachi CRT monitor hooked to GT710's VGA output, and I'm very much determined to get the text mode ("VGA mode 3") working on it -- it's my favorite mode of OS kernel development. After that, perhaps, I'll try HDMI too ;-)
Signature · 2 lines
Regards, Yuri
RISC-V (Unmatched): NVidia GK-208 card successfully initialized by U-Boot
In reply to Heinrich SchuchardtHi Heinrich, hi U-Boot RISC-V community,
Quoted 5 lines · Heinrich Schuchardt
Hello Yuri,
[...]
Were you able to show output on HDMI?
Yes :-)
(a slightly longer answer: but I could not get the x86 bios_emulator working with my card, despite all the efforts. So I went a completely different way)
I'm creating a blog series (8 posts) about my findings and about the end result. Can be seen at: https://r-tty.blogspot.com . Today I will also record a demonstration video.
To all SiFive Unmatched users (and also the users of PolarFire Icicle Kit!), who use the board with the PCIe video controller: please write me if you're interested. I'll be opening the sources of my "VideoBIOS" program soon (GPL2-licensed), after I finish some architectural work about QSOE's video console. The project will be available at: https://gitlab.com/bios-riscv
Signature · 2 lines
Regards, Yuri