Skip to contentSkip to conversation

How to use add-symbol-file in gdb after the program jumped to linux? (both when PC is physical and virtual)

DISCUSSION Chan Kim 1 participant 3 messages Jan 12, 2022 – Jan 13, 2022

lore
Share & tools
lore

Chan Kim

author
#0 lore raw

Hello experts,

I'm following linux boot-loading using u-boot (using SPL falcon mode, from RAM) on a qemu virtual machine (now linux started in real board too). The code jumped to linux kernel and because I have done add-symbol-file vmlinux 0x80081000 I can follow the kernel code step by step using gdb connected to the virtual machine. Actually I loaded the kernel image to 0x80080000 but I had to set the address to 0x80081000 to make the source code appear on the gdb correctly according to the PC value(I don't know why this difference of 0x1000 is needed).

Later I found the kernel sets up the page table (identity mapping and swap table) and jumps to __primary_switched and this is where pure kernel virtual address is used first time for the PC. This is where the call is made at the end of the head.S file.

ldr x8, =__primary_switched

adrp x0, __PHYS_OFFSET

br x8

In the symbol file (vmlinux, an elf file), the symbols before __primary_switched are all mapped at virtual addresses (starting with 0xffffffc0..... high addresses) but the gdb could follow the source even when the PC value was using physical address. (The PC was initially loaded with physical address of the kernel start and PC relative jumps were being used until it jumps to __primary_switched, mmu disabled or using identity mapping) So does this mean, in doing add-symbol-file only the offset of the symbols from the start of text matters?

Another question : I can follow the kernel source with gdb but after __primary_switched, I cannot see the source. The debugger doesn't show the correct source location according to the now kernel virtual PC value. Should I tell the debugger to use correct offset using add-symbol-file again? if so how?

I would be happy to hear any comment or advice.

Thank you!

Chan Kim

author
#1 lore raw

I found from gdb manual,

"add-symbol-file filename [ -readnow | -readnever ] [ -o offset ] [ textaddress ] [
-s section address ... ]
The add-symbol-file command reads additional symbol table information from the file filename. You would use this command when filename has been dynamically loaded (by some other means) into the program that is running. The textaddress parameter gives the memory address at which the file's text section has been loaded. You can additionally specify the base address of other
sections using an arbitrary number of '-s section address' pairs. If a section
is omitted, gdb will use its default addresses as found in filename. Any address
or textaddress can be given as an expression. ..."

I changed my program a little bit to fix a problem. The readelf shows the .text section starting at ffffffc010080800.
So I adjusted the command to "add-symbol-file vmlinux 0x80000800" and gdb shows the kernel source correct after jump to linux.
Still it doesn't show me the source code after __primary_switched.

Section Headers:

  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .head.text        PROGBITS         ffffffc010080000  00010000
       0000000000000040  0000000000000000  AX       0     0     4
  [ 2] .text             PROGBITS         ffffffc010080800  00010800
       0000000000304370  0000000000000000  AX       0     0     2048
  [ 3] .rodata           PROGBITS         ffffffc010390000  00320000
   .... (skip) ...
  [12] .notes            NOTE             ffffffc01045be18  003ebe18
       000000000000003c  0000000000000000   A       0     0     4
  [13] .init.text        PROGBITS         ffffffc010470000  003f0000
       0000000000027ec8  0000000000000000  AX       0     0     4
  [14] .exit.text        PROGBITS         ffffffc010497ec8  00417ec8
       000000000000046c  0000000000000000  AX       0     0     4

Since '__primary_switched' resides in section .init.text, I tried adding "-s .init.text 0xffffffc010470000" or "-s .init_text 0x803ef800"(physcial address) to the add-symbol-file command to no avail. Is my command wrong? Or could this be from page table (virtual -> Physical) problem because I see synchronous exception right after I enter __primary_switched (I see PC value has become 0x200. If the exception vector is located in 0x0, this is the vector entry for synch exception like undefined instruction. I should also check the vector base address has not been set correctly.)

Any comment or advice will be deeply appreciated.
Thank you!

Chan Kim

Quoted 64 lines

-----Original Message-----
From: U-Boot <[email protected]> On Behalf Of Chan Kim
Sent: Wednesday, January 12, 2022 4:43 PM
To: U-Boot Mailing List <[email protected]>
Subject: How to use add-symbol-file in gdb after the program jumped to linux? (both when PC is physical and virtual)

Hello experts,

I'm following linux boot-loading using u-boot (using SPL falcon mode, from RAM) on a qemu virtual machine (now linux started in real board too). The code jumped to linux kernel and because I have done add-symbol-file vmlinux 0x80081000 I can follow the kernel code step by step using gdb connected to the virtual machine. Actually I loaded the kernel image to 0x80080000 but I had to set the address to 0x80081000 to make the source code appear on the gdb correctly according to the PC value(I don't know why this difference of
0x1000 is needed).

Later I found the kernel sets up the page table (identity mapping and swap table) and jumps to __primary_switched and this is where pure kernel virtual address is used first time for the PC. This is where the call is made at the end of the head.S file.

ldr x8, =__primary_switched

adrp x0, __PHYS_OFFSET

br x8

In the symbol file (vmlinux, an elf file), the symbols before __primary_switched are all mapped at virtual addresses (starting with 0xffffffc0..... high addresses) but the gdb could follow the source even when the PC value was using physical address. (The PC was initially loaded with physical address of the kernel start and PC relative jumps were being used until it jumps to __primary_switched, mmu disabled or using identity
mapping) So does this mean, in doing add-symbol-file only the offset of the symbols from the start of text matters?

Another question : I can follow the kernel source with gdb but after __primary_switched, I cannot see the source. The debugger doesn't show the correct source location according to the now kernel virtual PC value. Should I tell the debugger to use correct offset using add-symbol-file again? if so how?

I would be happy to hear any comment or advice.

Thank you!

Chan Kim

author
#2 lore raw
In reply to Chan Kim

I found my kernel load address was wrong (__PHYS_OFFSET was below physical ddr address start).
After fixing it the PC increments normally with kernel virtual address and I should just apply the add-symbol-file command using the virtual address. With this command(with new kernel image), I could follow the source using qemu and gdb step-by-step after __primary_switched. add-symbol-file images/vmlinux 0xffffffc010080800 -s .head.text 0xffffffc010080000 -s .init.text 0xffffffc010470000
Hope this helps someone later.
Thanks!
Chan Kim

Quoted 9 lines

-----Original Message-----
From: Chan Kim <[email protected]>
Sent: Wednesday, January 12, 2022 5:31 PM
To: 'U-Boot Mailing List' <[email protected]>
Subject: RE: How to use add-symbol-file in gdb after the program jumped to linux? (both when PC is physical and virtual)

I found from gdb manual,

Quoted 8 lines

"add-symbol-file filename [ -readnow | -readnever ] [ -o offset ] [ textaddress ] [ -s section address ... ] The add-symbol-file command reads additional symbol table information from the file filename. You would use this command when filename has been dynamically loaded (by some other means) into the program that is running.
The textaddress parameter gives the memory address at which the file's text section has been loaded. You can additionally specify the base address of other sections using an arbitrary number of '-s section

address'

Quoted 37 lines

pairs. If a section is omitted, gdb will use its default addresses as found in filename. Any address or textaddress can be given as an expression. ..."

I changed my program a little bit to fix a problem. The readelf shows the .text section starting at ffffffc010080800.
So I adjusted the command to "add-symbol-file vmlinux 0x80000800" and gdb shows the kernel source correct after jump to linux.
Still it doesn't show me the source code after __primary_switched.

Section Headers:

  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .head.text        PROGBITS         ffffffc010080000  00010000
       0000000000000040  0000000000000000  AX       0     0     4
  [ 2] .text             PROGBITS         ffffffc010080800  00010800
       0000000000304370  0000000000000000  AX       0     0     2048
  [ 3] .rodata           PROGBITS         ffffffc010390000  00320000
   .... (skip) ...
  [12] .notes            NOTE             ffffffc01045be18  003ebe18
       000000000000003c  0000000000000000   A       0     0     4
  [13] .init.text        PROGBITS         ffffffc010470000  003f0000
       0000000000027ec8  0000000000000000  AX       0     0     4
  [14] .exit.text        PROGBITS         ffffffc010497ec8  00417ec8
       000000000000046c  0000000000000000  AX       0     0     4

Since '__primary_switched' resides in section .init.text, I tried adding "-s .init.text 0xffffffc010470000" or "-s .init_text 0x803ef800"(physcial address) to the add-symbol-file command to no avail. Is my command wrong? Or could this be from page table (virtual -> Physical) problem because I see synchronous exception right after I enter __primary_switched (I see PC value has become 0x200. If the exception vector is located in 0x0, this is the vector entry for synch exception like undefined instruction. I should also check the vector base address has not been set correctly.)

Quoted 57 lines

Any comment or advice will be deeply appreciated.
Thank you!

Chan Kim

Quoted 52 lines

-----Original Message-----
From: U-Boot <[email protected]> On Behalf Of Chan Kim
Sent: Wednesday, January 12, 2022 4:43 PM
To: U-Boot Mailing List <[email protected]>
Subject: How to use add-symbol-file in gdb after the program jumped to linux? (both when PC is physical and virtual)

Hello experts,

I'm following linux boot-loading using u-boot (using SPL falcon mode, from
RAM) on a qemu virtual machine (now linux started in real board too). The code jumped to linux kernel and because I have done add-symbol-file vmlinux 0x80081000 I can follow the kernel code step by step using gdb connected to the virtual machine. Actually I loaded the kernel image to
0x80080000 but I had to set the address to 0x80081000 to make the source code appear on the gdb correctly according to the PC value(I don't know why this difference of
0x1000 is needed).

Later I found the kernel sets up the page table (identity mapping and swap
table) and jumps to __primary_switched and this is where pure kernel virtual address is used first time for the PC. This is where the call is made at the end of the head.S file.

ldr x8, =__primary_switched

adrp x0, __PHYS_OFFSET

br x8

In the symbol file (vmlinux, an elf file), the symbols before __primary_switched are all mapped at virtual addresses (starting with 0xffffffc0..... high addresses) but the gdb could follow the source even when the PC value was using physical address. (The PC was initially loaded with physical address of the kernel start and PC relative jumps were being used until it jumps to __primary_switched, mmu disabled or using identity
mapping) So does this mean, in doing add-symbol-file only the offset of the symbols from the start of text matters?

Another question : I can follow the kernel source with gdb but after __primary_switched, I cannot see the source. The debugger doesn't show the correct source location according to the now kernel virtual PC

value.

Quoted 15 lines
Quoted 15 lines

Should I tell the debugger to use correct offset using add-symbol-file again? if so how?

I would be happy to hear any comment or advice.

Thank you!

Chan Kim