Fix symbol lookup for mips64* targets

For 64 bit targets, lookup_symbol() compares a 64-bit target address
with a 32 bit symbol address. This only works for addresses less than 2^32.

MIPS64 kernels use addresses larger than 0xffffffff80000000, so qemu.log never shows symbolic names.

My patch is a workaround which works with Qemu's 32 bit address hack. Please apply it to Qemu trunk.

Maybe a better solution would use symbol addresses without shortening them to 32 bits.

Quoted 4 lines · Stefan Weil

For 64 bit targets, lookup_symbol() compares a 64-bit target address with a 32 bit symbol address. This only works for addresses less than 2^32.

Quoted 5 lines

MIPS64 kernels use addresses larger than 0xffffffff80000000, so qemu.log never shows symbolic names.

My patch is a workaround which works with Qemu's 32 bit address hack. Please apply it to Qemu trunk.

This applies to all architectures, not just MIPS64, so it need not be conditional to TARGET_MIPS64.

Quoted 2 lines

Maybe a better solution would use symbol addresses without shortening them to 32 bits.

Yes. That would mean using elf_sym instead of Elf32_sym in places where symbols are used and removing the SZ==64 hacks in elf_ops.h.

Quoted 6 lines · Blue Swirl

Quoted 4 lines · Stefan Weil

For 64 bit targets, lookup_symbol() compares a 64-bit target address with a 32 bit symbol address. This only works for addresses less than 2^32.

Quoted 13 lines

Quoted 6 lines

MIPS64 kernels use addresses larger than 0xffffffff80000000, so qemu.log never shows symbolic names.

My patch is a workaround which works with Qemu's 32 bit address hack. Please apply it to Qemu trunk.

This applies to all architectures, not just MIPS64, so it need not be conditional to TARGET_MIPS64.

Quoted 3 lines

Maybe a better solution would use symbol addresses without shortening them to 32 bits.

Here is the better solution (at least I hope so).

Quoted 4 lines

Yes. That would mean using elf_sym instead of Elf32_sym in places where symbols are used and removing the SZ==64 hacks in elf_ops.h.

So I did. Here is a short summary of my new patch:

* Use function pointers for symbol lookup (currently for elf32 and elf64, could be expanded).
This also fixes the bug with mips elf64 symbols in current Qemu trunk.

* Use quicksort and binary search for symbol lookup.

* Remove unneeded entries from symbol table.
This reduced a typical table size (linux mips kernel) from 1764487 to
11656 entries.

* In disas.c, the patch also fixes some warnings from old fashioned function prototypes.

In loader.c, two defines control some compile time options (could be removed in
production code):
#define CONFIG_BINARY_SYMBOL_SEARCH
#define CONFIG_REDUCE_SYMBOL_TABLE

I tested the new code using 32 bit and 64 bit linux mips kernels and Qemu logging (-d in_asm).
The speed improvement is extremely large - both because of the much smaller table and
the binary search.

Stefan

The current patch only supports system emulation.
User emulation needs more fixes to compile again.

Quoted 27 lines · Stefan Weil

Here is a short summary of my new patch:

* Use function pointers for symbol lookup (currently for elf32 and elf64, could be expanded).
This also fixes the bug with mips elf64 symbols in current Qemu trunk.

* Use quicksort and binary search for symbol lookup.

* Remove unneeded entries from symbol table.
This reduced a typical table size (linux mips kernel) from 1764487 to 11656 entries.

* In disas.c, the patch also fixes some warnings from old fashioned function prototypes.

In loader.c, two defines control some compile time options (could be removed in
production code):
#define CONFIG_BINARY_SYMBOL_SEARCH
#define CONFIG_REDUCE_SYMBOL_TABLE

I tested the new code using 32 bit and 64 bit linux mips kernels and Qemu logging (-d in_asm).
The speed improvement is extremely large - both because of the much smaller table and
the binary search.

Quoted 2 lines

Stefan

Please note:

The current patch only supports system emulation.
User emulation needs more fixes to compile again.

Stefan

Quoted 8 lines

Quoted 6 lines · Stefan Weil

Here is a short summary of my new patch:

* Use function pointers for symbol lookup (currently for elf32 and elf64, could be expanded).
This also fixes the bug with mips elf64 symbols in current Qemu trunk.

Quoted 8 lines

Quoted 8 lines

* Use quicksort and binary search for symbol lookup.

* Remove unneeded entries from symbol table.
This reduced a typical table size (linux mips kernel) from 1764487 to 11656 entries.

* In disas.c, the patch also fixes some warnings from old fashioned function prototypes.

Quoted 13 lines

Quoted 13 lines

In loader.c, two defines control some compile time options (could be removed in
production code):
#define CONFIG_BINARY_SYMBOL_SEARCH
#define CONFIG_REDUCE_SYMBOL_TABLE

I tested the new code using 32 bit and 64 bit linux mips kernels and Qemu logging (-d in_asm).
The speed improvement is extremely large - both because of the much smaller table and
the binary search.

Quoted 10 lines

Quoted 2 lines

Stefan

Please note:

The current patch only supports system emulation.
User emulation needs more fixes to compile again.

Stefan

Here is an updated patch which now supports binary symbol lookup for both system emulation and user emulation.

User emulation was tested using qemu-x86_64 and a simple application with symbol information.

The new patch no longer includes compile time options for the old linear symbol search.

I hope this new patch will be included in Qemu trunk.

So I did. Here is a short summary of my new patch:

* Use function pointers for symbol lookup (currently for elf32 and elf64, could be expanded).
This also fixes the bug with mips elf64 symbols in current Qemu trunk.

* Use quicksort and binary search for symbol lookup.

* Remove unneeded entries from symbol table.
This reduced a typical table size (linux mips kernel) from 1764487 to
11656 entries.

* In disas.c, the patch also fixes some warnings from old fashioned function prototypes.

In loader.c, two defines control some compile time options (could be removed in
production code):
#define CONFIG_BINARY_SYMBOL_SEARCH
#define CONFIG_REDUCE_SYMBOL_TABLE

I tested the new code using 32 bit and 64 bit linux mips kernels and Qemu logging (-d in_asm).
The speed improvement is extremely large - both because of the much smaller table and
the binary search.

For 64 bit targets, lookup_symbol() compares a 64-bit target address with a 32 bit symbol address. This only works for addresses less than 2^32.

MIPS64 kernels use addresses larger than 0xffffffff80000000, so qemu.log never shows symbolic names.

My patch is a workaround which works with Qemu's 32 bit address hack. Please apply it to Qemu trunk.

Maybe a better solution would use symbol addresses without shortening them to 32 bits.

Regards
Stefan

Fix symbol lookup for mips64* targets.

Signed-off-by Stefan Weil <[email protected]>

Index: disas.c
===================================================================

Quoted 27 lines · Stefan Weil

For 64 bit targets, lookup_symbol() compares a 64-bit target address with a 32 bit symbol address. This only works for addresses less than 2^32.

MIPS64 kernels use addresses larger than 0xffffffff80000000, so qemu.log never shows symbolic names.

My patch is a workaround which works with Qemu's 32 bit address hack. Please apply it to Qemu trunk.

Maybe a better solution would use symbol addresses without shortening them to 32 bits.

Regards
Stefan

Fix symbol lookup for mips64* targets.

Signed-off-by Stefan Weil <[email protected]>

Index: disas.c
===================================================================

disas.c line 315

Quoted 7 lines

disas.c +3 -0

@@ -309,6 +309,11 @@
     struct syminfo *s;
     target_ulong addr;
 
+#if defined(TARGET_MIPS64)
+    /* Adresses in syminfos are 32 bit values. */
+    orig_addr &= 0xffffffff;
 

This isn't true in general, not all MIPS64 kernels use the -msym32 optimization.

Thiemo

add a note to the TODO list:
- store symbol addresses in full size (important for 64 bit targets)

On Fri, Oct 3, 2008 at 12:09 PM, Stefan Weil <[email protected]> wrote:
[...]

Quoted 2 lines

add a note to the TODO list:
- store symbol addresses in full size (important for 64 bit targets)

And while you're at it add a note saying using linked lists for that
is killing performance :-)

I have a hash-table based implementation somewhere if someone wants to take a look.

Quoted 41 lines · Thiemo Seufer

Quoted 27 lines · Stefan Weil

For 64 bit targets, lookup_symbol() compares a 64-bit target address with a 32 bit symbol address. This only works for addresses less than 2^32.

MIPS64 kernels use addresses larger than 0xffffffff80000000, so qemu.log never shows symbolic names.

My patch is a workaround which works with Qemu's 32 bit address hack. Please apply it to Qemu trunk.

Maybe a better solution would use symbol addresses without shortening them to 32 bits.

Regards
Stefan

Fix symbol lookup for mips64* targets.

Signed-off-by Stefan Weil <[email protected]>

Index: disas.c
===================================================================

disas.c (Arbeitskopie) line 315

Quoted 7 lines

disas.c (Arbeitskopie) renamed from disas.c (Revision 5400) +3 -0

@@ -309,6 +309,11 @@
     struct syminfo *s;
     target_ulong addr;
 
+#if defined(TARGET_MIPS64)
+    /* Adresses in syminfos are 32 bit values. */
+    orig_addr &= 0xffffffff;
 

This isn't true in general, not all MIPS64 kernels use the -msym32 optimization.

Quoted 1 line

Thiemo

Nor does my MIPS64 kernel: it uses 64 bit symbol addresses.

The current Qemu loads these addresses, reduces them to 32 bit and stores the reduced 32 bit addresses. So the 32 bit addresses are made by Qemu. They have nothing to do with kernel compile flags.

In lookup_symbol, Qemu compares real 64 bit addresses to the reduced 32 bit addresses which does not work for addresses larger than 2^32. x86_64 kernels have no problem because they have small addresses. MIPS64 kernels have a problem because they have large addresses. I don't know whether ppc64 and sparc64 work, maybe someone who has such kernels can do a "nm vmlinux".

My patch is only a workaround which reduces the real 64 bit addresses to 32 bit, too.

I suggest to apply it to Qemu trunk (because it is small and it improves the current situation) and to add a note to the TODO list:
- store symbol addresses in full size (important for 64 bit targets)