Skip to contentSkip to conversation

remove smaller slots if registering a bigger one

PATCH In review Glauber Costa 3 participants 10 messages Feb 11, 2009

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 remove smaller slots if registering a bigger one

It's like a shark eating a bunch of small fishes:
in some situations (vga linear frame buffer mapping, for example), we need to register a new slot in place of older, smaller ones. This patch handles this case

---
 kvm-all.c |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)
kvm-all.c +10 -0
index 9fb295c..53aca0a 100644
@@ -582,6 +582,16 @@ void kvm_set_phys_mem(target_phys_addr_t start_addr,
                 kvm_set_phys_mem(mem_start, mem_size, mem_offset);
 
             return;
+        } else if (start_addr <= mem->start_addr &&
+                   (start_addr + size) >= (mem->start_addr +
+                                           mem->memory_size)) {
+            KVMSlot slot;
+            /* unregister whole slot */
+            memcpy(&slot, mem, sizeof(slot));
+            mem->memory_size = 0;
+            kvm_set_user_memory_region(s, mem);
+
+            kvm_set_phys_mem(start_addr, size, phys_offset);
         } else {
             printf("Registering overlapping slot\n");
             abort();
Signature · 1 line
1.5.6.5
#1 lore raw
Quoted 18 lines · Glauber Costa

It's like a shark eating a bunch of small fishes:
in some situations (vga linear frame buffer mapping, for example), we need to register a new slot in place of older, smaller ones. This patch handles this case

---
 kvm-all.c |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)
kvm-all.c +0 -0
index 9fb295c..53aca0a 100644
@@ -582,6 +582,16 @@ void kvm_set_phys_mem(target_phys_addr_t start_addr,
                 kvm_set_phys_mem(mem_start, mem_size, mem_offset);
 
             return;

kvm-all.c void kvm_set_phys_mem(target_phys_addr_t start_addr, · line 594

Quoted 10 lines
kvm-all.c +10 -0
+        } else if (start_addr <= mem->start_addr &&
+                   (start_addr + size) >= (mem->start_addr +
+                                           mem->memory_size)) {
+            KVMSlot slot;
+            /* unregister whole slot */
+            memcpy(&slot, mem, sizeof(slot));
+            mem->memory_size = 0;
+            kvm_set_user_memory_region(s, mem);
+
+            kvm_set_phys_mem(start_addr, size, phys_offset);

That may solve some problems, but...

kvm-all.c void kvm_set_phys_mem(target_phys_addr_t start_addr, · line 597

Quoted 3 lines
kvm-all.c +0 -0
         } else {
             printf("Registering overlapping slot\n");
             abort();
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

...as long as this line exists, issues will remain. IIRC, the mapping the i440 tries to re-establish after reboot will hit this case.

BTW, I found the unposted patch below in my attic, maybe you can comment on it (if it makes sense, I'll properly repost with signed-off).

Thanks,
Jan

---------->

kvm: cleanup unmap condition in kvm_set_phys_mem

Testing for TLB_MMIO on unmap makes no sense as A) that flag belongs to CPUTLBEntry and not to io_memory slots or physical addresses and B) we already use a different condition before mapping. So make this test consistent.

kvm-all.c +1 -1
index 9fb295c..c0481a0 100644
@@ -540,7 +540,7 @@ void kvm_set_phys_mem(target_phys_addr_t start_addr,
 
     mem = kvm_lookup_slot(s, start_addr);
     if (mem) {
-        if ((flags == IO_MEM_UNASSIGNED) || (flags >= TLB_MMIO)) {
+        if (flags >= IO_MEM_UNASSIGNED) {
             mem->memory_size = 0;
             mem->start_addr = start_addr;
             mem->phys_offset = 0;
#2 lore raw
Quoted 32 lines · Jan Kiszka
Quoted 18 lines · Glauber Costa

It's like a shark eating a bunch of small fishes:
in some situations (vga linear frame buffer mapping, for example), we need to register a new slot in place of older, smaller ones. This patch handles this case

---
 kvm-all.c |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)
kvm-all.c +0 -0
index 9fb295c..53aca0a 100644
@@ -582,6 +582,16 @@ void kvm_set_phys_mem(target_phys_addr_t start_addr,
                 kvm_set_phys_mem(mem_start, mem_size, mem_offset);
 
             return;

kvm-all.c void kvm_set_phys_mem(target_phys_addr_t start_addr, · line 594

Quoted 10 lines
kvm-all.c +10 -0
+        } else if (start_addr <= mem->start_addr &&
+                   (start_addr + size) >= (mem->start_addr +
+                                           mem->memory_size)) {
+            KVMSlot slot;
+            /* unregister whole slot */
+            memcpy(&slot, mem, sizeof(slot));
+            mem->memory_size = 0;
+            kvm_set_user_memory_region(s, mem);
+
+            kvm_set_phys_mem(start_addr, size, phys_offset);

That may solve some problems, but...

Quoted 6 lines

kvm-all.c void kvm_set_phys_mem(target_phys_addr_t start_addr, · line 597

Quoted 3 lines
kvm-all.c +0 -0
         } else {
             printf("Registering overlapping slot\n");
             abort();
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

...as long as this line exists, issues will remain. IIRC, the mapping the i440 tries to re-establish after reboot will hit this case.

Which is fine. I'd prefer it to be here, so we can analyse it case by case. The old memory code for kvm was totally messy, in part because we tried to hug the world at once, with some code paths that were almost never hit.

Slot management can easily get very complicated. and trying to come up with a solution that accounts for all problems at once may backfire on us.

Quoted 4 lines

BTW, I found the unposted patch below in my attic, maybe you can comment on it (if it makes sense, I'll properly repost with signed-off).

I don't believe it makes much sense.

kvm-all.c void kvm_set_phys_mem(target_phys_addr_t start_addr, · line 546

Quoted 9 lines
kvm-all.c +1 -1
@@ -540,7 +540,7 @@ void kvm_set_phys_mem(target_phys_addr_t start_addr,
 
     mem = kvm_lookup_slot(s, start_addr);
     if (mem) {
-        if ((flags == IO_MEM_UNASSIGNED) || (flags >= TLB_MMIO)) {
+        if (flags >= IO_MEM_UNASSIGNED) {
             mem->memory_size = 0;
             mem->start_addr = start_addr;
             mem->phys_offset = 0;

vga seem to be a heavy user of this kind of construct. We map some piece of memory as RAM, and later on, it becomes an mmio region again. In this case, we have to delete it from kvm slot list.

Now, if you remember the last memory patches I sent, it actually removes this line. However, this is because in that alternative, we were tracking mmio regions in qemu, but not in kvm. so flags >= TLB_MMIO would just delete it from the kernel mapping, but qemu would not forget about it.

This is to show that I believe that it might be possible to handle mmio regions slightly different, but I don't think just dropping this line would help much.

#3 lore raw
Quoted 38 lines · Glauber Costa
Quoted 31 lines · Jan Kiszka
Quoted 18 lines · Glauber Costa

It's like a shark eating a bunch of small fishes:
in some situations (vga linear frame buffer mapping, for example), we need to register a new slot in place of older, smaller ones. This patch handles this case

---
 kvm-all.c |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)
kvm-all.c +0 -0
index 9fb295c..53aca0a 100644
@@ -582,6 +582,16 @@ void kvm_set_phys_mem(target_phys_addr_t start_addr,
                 kvm_set_phys_mem(mem_start, mem_size, mem_offset);
 
             return;

kvm-all.c void kvm_set_phys_mem(target_phys_addr_t start_addr, · line 594

Quoted 10 lines
kvm-all.c +10 -0
+        } else if (start_addr <= mem->start_addr &&
+                   (start_addr + size) >= (mem->start_addr +
+                                           mem->memory_size)) {
+            KVMSlot slot;
+            /* unregister whole slot */
+            memcpy(&slot, mem, sizeof(slot));
+            mem->memory_size = 0;
+            kvm_set_user_memory_region(s, mem);
+
+            kvm_set_phys_mem(start_addr, size, phys_offset);

That may solve some problems, but...

Quoted 6 lines

kvm-all.c void kvm_set_phys_mem(target_phys_addr_t start_addr, · line 597

Quoted 3 lines
kvm-all.c +0 -0
         } else {
             printf("Registering overlapping slot\n");
             abort();
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

...as long as this line exists, issues will remain. IIRC, the mapping the i440 tries to re-establish after reboot will hit this case.

Quoted 7 lines

Which is fine. I'd prefer it to be here, so we can analyse it case by case. The old memory code for kvm was totally messy, in part because we tried to hug the world at once, with some code paths that were almost never hit.

Slot management can easily get very complicated. and trying to come up with a solution that accounts for all problems at once may backfire on us.

Well, then "fix" all users...

Quoted 19 lines
Quoted 4 lines

BTW, I found the unposted patch below in my attic, maybe you can comment on it (if it makes sense, I'll properly repost with signed-off).

I don't believe it makes much sense.

kvm-all.c void kvm_set_phys_mem(target_phys_addr_t start_addr, · line 546

Quoted 9 lines
kvm-all.c +1 -1
@@ -540,7 +540,7 @@ void kvm_set_phys_mem(target_phys_addr_t start_addr,
 
     mem = kvm_lookup_slot(s, start_addr);
     if (mem) {
-        if ((flags == IO_MEM_UNASSIGNED) || (flags >= TLB_MMIO)) {
+        if (flags >= IO_MEM_UNASSIGNED) {
             mem->memory_size = 0;
             mem->start_addr = start_addr;
             mem->phys_offset = 0;

vga seem to be a heavy user of this kind of construct. We map some piece of memory as RAM, and later on, it becomes an mmio region again. In this case, we have to delete it from kvm slot list.

Quoted 8 lines

Now, if you remember the last memory patches I sent, it actually removes this line. However, this is because in that alternative, we were tracking mmio regions in qemu, but not in kvm. so flags >= TLB_MMIO would just delete it from the kernel mapping, but qemu would not forget about it.

This is to show that I believe that it might be possible to handle mmio regions slightly different, but I don't think just dropping this line would help much.

What confused me most here - and still does - is that TLB_* flags make semantically no sense to me in the physical memory registering/unregistering context. They should only show up in CPUTLBEntry, no? But maybe I'm overseeing some strange relation.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#4 lore raw
Quoted 43 lines · Jan Kiszka
Quoted 42 lines · Glauber Costa
Quoted 31 lines · Jan Kiszka
Quoted 18 lines · Glauber Costa

It's like a shark eating a bunch of small fishes:
in some situations (vga linear frame buffer mapping, for example), we need to register a new slot in place of older, smaller ones. This patch handles this case

---
 kvm-all.c |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)
kvm-all.c +0 -0
index 9fb295c..53aca0a 100644
@@ -582,6 +582,16 @@ void kvm_set_phys_mem(target_phys_addr_t start_addr,
                 kvm_set_phys_mem(mem_start, mem_size, mem_offset);
 
             return;

kvm-all.c void kvm_set_phys_mem(target_phys_addr_t start_addr, · line 594

Quoted 10 lines
kvm-all.c +10 -0
+        } else if (start_addr <= mem->start_addr &&
+                   (start_addr + size) >= (mem->start_addr +
+                                           mem->memory_size)) {
+            KVMSlot slot;
+            /* unregister whole slot */
+            memcpy(&slot, mem, sizeof(slot));
+            mem->memory_size = 0;
+            kvm_set_user_memory_region(s, mem);
+
+            kvm_set_phys_mem(start_addr, size, phys_offset);

That may solve some problems, but...

Quoted 6 lines

kvm-all.c void kvm_set_phys_mem(target_phys_addr_t start_addr, · line 597

Quoted 3 lines
kvm-all.c +0 -0
         } else {
             printf("Registering overlapping slot\n");
             abort();
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

...as long as this line exists, issues will remain. IIRC, the mapping the i440 tries to re-establish after reboot will hit this case.

Which is fine. I'd prefer it to be here, so we can analyse it case by case. The old memory code for kvm was totally messy, in part because we tried to hug the world at once, with some code paths that were almost never hit.

Quoted 5 lines
Quoted 3 lines

Slot management can easily get very complicated. and trying to come up with a solution that accounts for all problems at once may backfire on us.

Well, then "fix" all users...

BTW, if you want to play with some problematic case, apply this and reboot a guest while using -enable-kvm:

vl.c +2 -0
index ce80690..d53611e 100644
@@ -3557,6 +3557,8 @@ void qemu_system_reset(void)
     for(re = first_reset_entry; re != NULL; re = re->next) {
         re->func(re->opaque);
     }
+    if (kvm_enabled())
+        kvm_sync_vcpus();
 }
 
 void qemu_system_reset_request(void)

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#5 lore raw
In reply to Jan Kiszka
Quoted 6 lines
Quoted 6 lines

Now, if you remember the last memory patches I sent, it actually removes this line. However, this is because in that alternative, we were tracking mmio regions in qemu, but not in kvm. so flags >= TLB_MMIO would just delete it from the kernel mapping, but qemu would not forget about it.

Quoted 9 lines
Quoted 3 lines

This is to show that I believe that it might be possible to handle mmio regions slightly different, but I don't think just dropping this line would help much.

What confused me most here - and still does - is that TLB_* flags make semantically no sense to me in the physical memory registering/unregistering context. They should only show up in CPUTLBEntry, no? But maybe I'm overseeing some strange relation.

I might be wrong, but I believe it's used quite often in normal memory to track pieces of memory in which we do mmio.

#6 lore raw
In reply to Jan Kiszka
Quoted 11 lines · Jan Kiszka
Quoted 8 lines

Which is fine. I'd prefer it to be here, so we can analyse it case by case. The old memory code for kvm was totally messy, in part because we tried to hug the world at once, with some code paths that were almost never hit.

Slot management can easily get very complicated. and trying to come up with a solution that accounts for all problems at once may backfire on us.

Well, then "fix" all users...

Making memory management in qemu symmetric (any cpu_register_physical_memory() must be followed by a cpu_unregister_physical_memory() with the same parameters) is a good idea.

Signature · 1 line
error compiling committee.c: too many arguments to function
#7 lore raw
Quoted 13 lines · Avi Kivity
Quoted 12 lines · Jan Kiszka
Quoted 11 lines

Which is fine. I'd prefer it to be here, so we can analyse it case by case.
The old memory code for kvm was totally messy, in part because we tried to
hug the world at once, with some code paths that were almost never hit.

Slot management can easily get very complicated. and trying to come up with a solution that accounts for all problems at once may backfire on us.

Quoted 7 lines
Quoted 2 lines

Well, then "fix" all users...

Making memory management in qemu symmetric (any cpu_register_physical_memory() must be followed by a cpu_unregister_physical_memory() with the same parameters) is a good idea.

Then all devices should track their - potentially guest-defined - mappings and revert them on reset? I think handling this in the lower layer is more convenient.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux
#8 lore raw
Quoted 10 lines · Jan Kiszka
Quoted 5 lines

Making memory management in qemu symmetric (any cpu_register_physical_memory() must be followed by a cpu_unregister_physical_memory() with the same parameters) is a good idea.

Then all devices should track their - potentially guest-defined - mappings and revert them on reset? I think handling this in the lower layer is more convenient.

Most of these mappings (at least in the PC world) are PCI mappings, no? In which case the PCI layer already tracks everything.

There's an issue if the guest programs conflicting BARs. Currently qemu has a latest-wins policy, but that's not necessarily the right thing.

Signature · 1 line
error compiling committee.c: too many arguments to function
#9 lore raw
In reply to Jan Kiszka
Quoted 5 lines · Jan Kiszka
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

...as long as this line exists, issues will remain. IIRC, the mapping the i440 tries to re-establish after reboot will hit this case.

I agree. Slot management is similar to vma management in the kernel. We're probably better off keeping the slot list sorted and implementing all possible map/unmap cases wrt partial overlap.

Signature · 1 line
error compiling committee.c: too many arguments to function