Skip to contentSkip to conversation

Reduce generated code by 3% by increasing MMU indices

RFC PATCH In review Helge Deller 2 participants 7 messages Aug 5, 2023

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 Reduce generated code by 3% by increasing MMU indices

This is a RFC, in which I want to bring up an option on how to reduce the generated code size by in average 6 bytes for *every* memory-access inside the guest. This applies to nearly every guest target on a x86 host. Other hosts may benefit too, but I didn't checked.

In include/exec/cpu-defs.h the maximum number of MMU modes is defined as:
#define NB_MMU_MODES 16

Most targets use less MMU indexes and define them similiar to this:

#define MMU_KERNEL_IDX   0
#define MMU_USER_IDX     1
#define MMU_PHYS_IDX     2

or, e.g. on ppc, by a function which gives some number in the range 0-7:

static inline int cpu_mmu_index(CPUPPCState *env, bool ifetch) {
#ifdef CONFIG_USER_ONLY
return MMU_USER_IDX; /* MMU_USER_IDX is 0 */

#else
    return (env->hflags >> (ifetch ? HFLAGS_IMMU_IDX : HFLAGS_DMMU_IDX)) & 7;

#endif
}

When looking at the generated code for every memory-access in the guest, the tcg generates a CPU TLB lookup in the fast path, e.g. for a x86 guest on x86 host (only relevant part shown below):

IN:
0x000ebdf5: 8b 04 24 movl (%esp), %eax

OUT:
...

0x003619:  48 23 bd 10 ff ff ff     andq     -0xf0(%rbp), %rdi
0x003620:  48 03 bd 18 ff ff ff     addq     -0xe8(%rbp), %rdi

...

As can be seen, the TLB mask entry is accessed with a negative offset.

By re-defining the MMU indices to become 15,14,13 instead:

#define MMU_KERNEL_IDX   (NB_MMU_MODES - 1)
#define MMU_USER_IDX     (NB_MMU_MODES - 2)
#define MMU_PHYS_IDX     (NB_MMU_MODES - 3)

the (negative) offset is smaller, and the x86-64 tcg will generate a 4-byte (instead of 7-byte) instruction:
OUT:
...

0x003499:  48 23 7d c0              andq     -0x40(%rbp), %rdi
0x00349d:  48 03 7d c8              addq     -0x38(%rbp), %rdi

So, every memory acces in the guest saves 6 bytes (=2 * 3 bytes) of instruction code in the fast path.

Looking at the instruction address offsets (0x003619 vs. 0x003499) it already saved 0x180 (384 decimal) bytes.
The first instruction was at offset 0x000000, so an overall instruction size reduction of ~3% can be seen.

To reproduce I used this command:
./qemu-system-x86_64 -cdrom ./debian-12.1.0-amd64-netinst.iso \
-boot d -nographic -d in_asm,out_asm

Do we want to enable such an performance optimization? If so, I see two possibilities:

a) Re-define NB_MMU_MODES per target, with NB_MMU_MODES becoming the highest MMU index needed for that target. This will probably give another small performance improvement as the flush loop will become shorter.

b) Increase the MMU index per target as shown in the patch below. The patch below covers x86, ppc, alpha, hppa.
For arm, I believe it's sufficient to change ARM_MMU_IDX_M_PRIV=>0xf, ARM_MMU_IDX_M_NEGPRI=0xe, and ARM_MMU_IDX_M_S=>0xd.

Opinions?

Helge

target/i386/cpu.h +5 -5
index e0771a1043..d4aa6e7bee 100644
@@ -2251,11 +2251,11 @@ uint64_t cpu_get_tsc(CPUX86State *env);
 #define cpu_list x86_cpu_list
 
 /* MMU modes definitions */
-#define MMU_KSMAP_IDX   0
-#define MMU_USER_IDX    1
-#define MMU_KNOSMAP_IDX 2
-#define MMU_NESTED_IDX  3
-#define MMU_PHYS_IDX    4
+#define MMU_KSMAP_IDX   (NB_MMU_MODES - 1)
+#define MMU_USER_IDX    (NB_MMU_MODES - 2)
+#define MMU_KNOSMAP_IDX (NB_MMU_MODES - 3)
+#define MMU_NESTED_IDX  (NB_MMU_MODES - 4)
+#define MMU_PHYS_IDX    (NB_MMU_MODES - 5)
 
 static inline int cpu_mmu_index(CPUX86State *env, bool ifetch)
 {
target/ppc/cpu.h +3 -2
index 25fac9577a..a2a56781eb 100644
@@ -1474,13 +1474,14 @@ int ppc_dcr_write(ppc_dcr_t *dcr_env, int dcrn, uint32_t val);
 #define cpu_list ppc_cpu_list
 
 /* MMU modes definitions */
-#define MMU_USER_IDX 0
+#define MMU_USER_IDX (NB_MMU_MODES - 1)
 static inline int cpu_mmu_index(CPUPPCState *env, bool ifetch)
 {
 #ifdef CONFIG_USER_ONLY
     return MMU_USER_IDX;
 #else
-    return (env->hflags >> (ifetch ? HFLAGS_IMMU_IDX : HFLAGS_DMMU_IDX)) & 7;
+    return NB_MMU_MODES - 2
+           - ((env->hflags >> (ifetch ? HFLAGS_IMMU_IDX : HFLAGS_DMMU_IDX)) & 7);
 #endif
 }
 
target/alpha/cpu.h +3 -3
index 13306665af..f25cf33e25 100644
@@ -194,9 +194,9 @@ enum {
    PALcode cheats and uses the KSEG mapping for its code+data rather than
    physical addresses.  */
 
-#define MMU_KERNEL_IDX   0
-#define MMU_USER_IDX     1
-#define MMU_PHYS_IDX     2
+#define MMU_KERNEL_IDX   (NB_MMU_MODES - 1)
+#define MMU_USER_IDX     (NB_MMU_MODES - 2)
+#define MMU_PHYS_IDX     (NB_MMU_MODES - 3)
 
 typedef struct CPUArchState {
     uint64_t ir[31];
target/hppa/cpu.h +3 -3
index 75c5c0ccf7..1c09602d0b 100644
@@ -30,9 +30,9 @@
    basis.  It's probably easier to fall back to a strong memory model.  */
 #define TCG_GUEST_DEFAULT_MO        TCG_MO_ALL
 
-#define MMU_KERNEL_IDX   0
-#define MMU_USER_IDX     3
-#define MMU_PHYS_IDX     4
+#define MMU_KERNEL_IDX   (NB_MMU_MODES - 1)
+#define MMU_USER_IDX     (NB_MMU_MODES - 2)
+#define MMU_PHYS_IDX     (NB_MMU_MODES - 3)
 #define TARGET_INSN_START_EXTRA_WORDS 1
 
 /* Hardware exceptions, interrupts, faults, and traps.  */
#1 lore raw
Quoted 4 lines · Helge Deller

Do we want to enable such an performance optimization? If so, I see two possibilities:

a) Re-define NB_MMU_MODES per target

No, we've just gotten rid of per target definitions of NB_MMU_MODES, on the way to being able to support multiple targets simultaneously.

This only affects x86, and for only 6 bytes per memory access. While saving code size is a nice goal, I sincerely doubt you can measure any performance difference.

If there were a way to change no more than two lines of code, that would be fine. But otherwise I don't see this as being worth making the rest of the code base any more complex.

r~

#2 lore raw

* Richard Henderson <[email protected]>:

Quoted 8 lines · Helge Deller
Quoted 4 lines

Do we want to enable such an performance optimization? If so, I see two possibilities:

a) Re-define NB_MMU_MODES per target

No, we've just gotten rid of per target definitions of NB_MMU_MODES, on the way to being able to support multiple targets simultaneously.

Ok, I assume that answer :-)

Quoted 3 lines

This only affects x86, and for only 6 bytes per memory access. While saving code size is a nice goal, I sincerely doubt you can measure any performance difference.

Maybe. I don't know. I'm sure the gain is small, but the patch is small too.

Quoted 3 lines

If there were a way to change no more than two lines of code, that would be fine. But otherwise I don't see this as being worth making the rest of the code base any more complex.

Ok. What about that 6-line patch below for x86?
It's trivial and all what's needed for x86.
Btw, any index which is >= 9 will use the shorter code sequence.

Helge

target/i386/cpu.h +5 -5
index e0771a1043..3e71e666db 100644
@@ -2251,11 +2251,11 @@ uint64_t cpu_get_tsc(CPUX86State *env);
 #define cpu_list x86_cpu_list
 
 /* MMU modes definitions */
-#define MMU_KSMAP_IDX   0
-#define MMU_USER_IDX    1
-#define MMU_KNOSMAP_IDX 2
-#define MMU_NESTED_IDX  3
-#define MMU_PHYS_IDX    4
+#define MMU_KSMAP_IDX   11
+#define MMU_USER_IDX    12
+#define MMU_KNOSMAP_IDX 13
+#define MMU_NESTED_IDX  14
+#define MMU_PHYS_IDX    15
 
 static inline int cpu_mmu_index(CPUX86State *env, bool ifetch)
 {
#3 lore raw
Quoted 18 lines · Helge Deller
Quoted 3 lines

If there were a way to change no more than two lines of code, that would be fine. But otherwise I don't see this as being worth making the rest of the code base any more complex.

Ok. What about that 6-line patch below for x86?
It's trivial and all what's needed for x86.
Btw, any index which is >= 9 will use the shorter code sequence.

Helge

target/i386/cpu.h +0 -0
index e0771a1043..3e71e666db 100644
@@ -2251,11 +2251,11 @@ uint64_t cpu_get_tsc(CPUX86State *env);
  #define cpu_list x86_cpu_list
 
  /* MMU modes definitions */

target/i386/cpu.h uint64_t cpu_get_tsc(CPUX86State *env); · line 2258

Quoted 10 lines
target/i386/cpu.h +5 -5
-#define MMU_KSMAP_IDX   0
-#define MMU_USER_IDX    1
-#define MMU_KNOSMAP_IDX 2
-#define MMU_NESTED_IDX  3
-#define MMU_PHYS_IDX    4
+#define MMU_KSMAP_IDX   11
+#define MMU_USER_IDX    12
+#define MMU_KNOSMAP_IDX 13
+#define MMU_NESTED_IDX  14
+#define MMU_PHYS_IDX    15

No. The small patch would need to apply to all guests.

Perhaps something to handle indexing of CPUTLBDescFast, e.g.

static inline CPUTLBDescFast cputlb_fast(CPUTLB *tlb, unsigned idx)

{
    return &tlb->f[NB_MMU_MODES - 1 - idx];
}

There's already tlb_mask_table_ofs, which handles all tcg backends; you just need to adjust that and cputlb.c.

Introduce cputlb_fast with normal indexing in one patch, and then the second patch to invert the indexing may well be exactly two lines. :-)

r~

#4 lore raw
Quoted 16 lines · Richard Henderson
Quoted 15 lines · Helge Deller
Quoted 3 lines

If there were a way to change no more than two lines of code, that would be fine. But otherwise I don't see this as being worth making the rest of the code base any more complex.

Ok. What about that 6-line patch below for x86? It's trivial and all what's needed for x86. Btw, any index which is >= 9 will use the shorter code sequence.

Helge

target/i386/cpu.h index renamed from target/i386/cpu.h +0 -0

e0771a1043..3e71e666db 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -2251,11 +2251,11 @@ uint64_t cpu_get_tsc(CPUX86State *env); #define cpu_list x86_cpu_list

Quoted 7 lines
Quoted 5 lines

/* MMU modes definitions */ -#define MMU_KSMAP_IDX 0 -#define
MMU_USER_IDX 1 -#define MMU_KNOSMAP_IDX 2 -#define
MMU_NESTED_IDX 3 -#define MMU_PHYS_IDX 4 +#define MMU_KSMAP_IDX
11 +#define MMU_USER_IDX 12 +#define MMU_KNOSMAP_IDX 13 +#define
MMU_NESTED_IDX 14 +#define MMU_PHYS_IDX 15

No. The small patch would need to apply to all guests.

Yes.

Quoted 9 lines

Perhaps something to handle indexing of CPUTLBDescFast, e.g.

static inline CPUTLBDescFast cputlb_fast(CPUTLB *tlb, unsigned idx) { return &tlb->f[NB_MMU_MODES - 1 - idx]; }

There's already tlb_mask_table_ofs, which handles all tcg backends; you just need to adjust that and cputlb.c> Introduce cputlb_fast with normal indexing in one patch, and then the second patch to invert the indexing may well be exactly two lines. :-)

You're cheating :-)
But ok, that's an easy one and I can come up with both patches.

One last idea which came into my mind and which may be worth asking before I start to hack the patch above...:

include/exec/cpu-defs.h:
/* add some comment here why we use this transformation: */
#define MMU_INDEX(nr) (NB_MMU_MODES - 1 - (x))

target/*/cpu.h:
/* MMU modes definitions */
#define MMU_KSMAP_IDX MMU_INDEX(0)
#define MMU_USER_IDX MMU_INDEX(1)
#define MMU_KNOSMAP_IDX MMU_INDEX(2)
#define MMU_NESTED_IDX MMU_INDEX(3)
...

Downside:
- of course it's a lot more than the 2 lines you asked for Upsides:
- no additional subtaction at tcg compile time/runtime
- clear indication that this is an MMU index, easy to grep.
- easy to use

?

Helge

#5 lore raw
Quoted 55 lines · Helge Deller
Quoted 16 lines · Richard Henderson
Quoted 15 lines · Helge Deller
Quoted 3 lines

If there were a way to change no more than two lines of code, that would be fine.  But otherwise I don't see this as being worth making the rest of the code base any more complex.

Ok. What about that 6-line patch below for x86? It's trivial and all what's needed for x86. Btw, any index which is >= 9 will use the shorter code sequence.

Helge

target/i386/cpu.h index renamed from target/i386/cpu.h +0 -0

e0771a1043..3e71e666db 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -2251,11 +2251,11 @@ uint64_t cpu_get_tsc(CPUX86State *env); #define cpu_list x86_cpu_list

Quoted 7 lines
Quoted 5 lines

/* MMU modes definitions */ -#define MMU_KSMAP_IDX   0 -#define MMU_USER_IDX    1 -#define MMU_KNOSMAP_IDX 2 -#define MMU_NESTED_IDX  3 -#define MMU_PHYS_IDX    4 +#define MMU_KSMAP_IDX 11 +#define MMU_USER_IDX    12 +#define MMU_KNOSMAP_IDX 13 +#define MMU_NESTED_IDX  14 +#define MMU_PHYS_IDX    15

No.  The small patch would need to apply to all guests.

Yes.

Quoted 9 lines

Perhaps something to handle indexing of CPUTLBDescFast, e.g.

static inline CPUTLBDescFast cputlb_fast(CPUTLB *tlb, unsigned idx) { return &tlb->f[NB_MMU_MODES - 1 - idx]; }

There's already tlb_mask_table_ofs, which handles all tcg backends; you just need to adjust that and cputlb.c> Introduce cputlb_fast with normal indexing in one patch, and then the second patch to invert the indexing may well be exactly two lines.  :-)

You're cheating :-)
But ok, that's an easy one and I can come up with both patches.

One last idea which came into my mind and which may be worth asking before I start to hack the patch above...:

include/exec/cpu-defs.h:
/* add some comment here why we use this transformation: */ #define MMU_INDEX(nr)    (NB_MMU_MODES - 1 - (x))

target/*/cpu.h:
/* MMU modes definitions */
#define MMU_KSMAP_IDX   MMU_INDEX(0)
#define MMU_USER_IDX    MMU_INDEX(1)
#define MMU_KNOSMAP_IDX MMU_INDEX(2)
#define MMU_NESTED_IDX  MMU_INDEX(3)
...

Quoted 6 lines

Downside:
- of course it's a lot more than the 2 lines you asked for Upsides:
- no additional subtaction at tcg compile time/runtime
- clear indication that this is an MMU index, easy to grep.
- easy to use

and it's actually a 1-line patch as you requested :-)
similiar to your approach above (multiple preparation patches, one last patch which just changes
#define MMU_INDEX(nr) (nr)
to
#define MMU_INDEX(nr) (NB_MMU_MODES - 1 - (nr))

;-)

Helge

#6 lore raw
Quoted 63 lines · Helge Deller
Quoted 55 lines · Helge Deller
Quoted 16 lines · Richard Henderson
Quoted 15 lines · Helge Deller
Quoted 3 lines

If there were a way to change no more than two lines of code, that would be fine.  But otherwise I don't see this as being worth making the rest of the code base any more complex.

Ok. What about that 6-line patch below for x86? It's trivial and all what's needed for x86. Btw, any index which is >= 9 will use the shorter code sequence.

Helge

target/i386/cpu.h index renamed from target/i386/cpu.h +0 -0

e0771a1043..3e71e666db 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -2251,11 +2251,11 @@ uint64_t cpu_get_tsc(CPUX86State *env); #define cpu_list x86_cpu_list

Quoted 7 lines
Quoted 5 lines

/* MMU modes definitions */ -#define MMU_KSMAP_IDX   0 -#define MMU_USER_IDX    1 -#define MMU_KNOSMAP_IDX 2 -#define MMU_NESTED_IDX  3 -#define MMU_PHYS_IDX    4 +#define MMU_KSMAP_IDX 11 +#define MMU_USER_IDX    12 +#define MMU_KNOSMAP_IDX 13 +#define MMU_NESTED_IDX  14 +#define MMU_PHYS_IDX    15

No.  The small patch would need to apply to all guests.

Yes.

Quoted 9 lines

Perhaps something to handle indexing of CPUTLBDescFast, e.g.

static inline CPUTLBDescFast cputlb_fast(CPUTLB *tlb, unsigned idx) { return &tlb->f[NB_MMU_MODES - 1 - idx]; }

There's already tlb_mask_table_ofs, which handles all tcg backends; you just need to adjust that and cputlb.c> Introduce cputlb_fast with normal indexing in one patch, and then the second patch to invert the indexing may well be exactly two lines.  :-)

You're cheating :-)
But ok, that's an easy one and I can come up with both patches.

One last idea which came into my mind and which may be worth asking before I start to hack the patch above...:

include/exec/cpu-defs.h:
/* add some comment here why we use this transformation: */ #define MMU_INDEX(nr)    (NB_MMU_MODES - 1 - (x))

target/*/cpu.h:
/* MMU modes definitions */
#define MMU_KSMAP_IDX   MMU_INDEX(0)
#define MMU_USER_IDX    MMU_INDEX(1)
#define MMU_KNOSMAP_IDX MMU_INDEX(2)
#define MMU_NESTED_IDX  MMU_INDEX(3)
...

Quoted 6 lines

Downside:
- of course it's a lot more than the 2 lines you asked for Upsides:
- no additional subtaction at tcg compile time/runtime
- clear indication that this is an MMU index, easy to grep.
- easy to use

Quoted 8 lines

and it's actually a 1-line patch as you requested :-)
  similiar to your approach above (multiple preparation patches, one last patch which just changes
#define MMU_INDEX(nr)    (nr)
  to
#define MMU_INDEX(nr)    (NB_MMU_MODES - 1 - (nr))

;-)

:-) Plausible. With a go, anyway.

r~