Frame buffer corruptions with KVM >= 2.6.36
Hi,
I'm seeing quite frequent corruptions of the VESA frame buffer with Linux guests (vga=0x317) that are starting with KVM kernel modules of upcoming 2.6.36 (I'm currently running -rc7). Effects disappears when downgrading to kvm-kmod-2.6.35.6. Will see if I can bisect later, but maybe someone already has an idea or wants to reproduce (just run something like "find /" on one text console and witch to another one - text fragments will remain on the screen on every few switches).
Jan
Quoted 10 lines · Jan Kiszka
Hi,
I'm seeing quite frequent corruptions of the VESA frame buffer with Linux guests (vga=0x317) that are starting with KVM kernel modules of upcoming 2.6.36 (I'm currently running -rc7). Effects disappears when downgrading to kvm-kmod-2.6.35.6. Will see if I can bisect later, but maybe someone already has an idea or wants to reproduce (just run something like "find /" on one text console and witch to another one - text fragments will remain on the screen on every few switches).
Reproduces on kvm.git. I wonder what's going on.
Looks like vesafb uses the bios to switch the display start, so I expect a problem in qemu reacting to this.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 12 lines · Avi Kivity
Quoted 10 lines · Jan Kiszka
Hi,
I'm seeing quite frequent corruptions of the VESA frame buffer with Linux guests (vga=0x317) that are starting with KVM kernel modules of upcoming 2.6.36 (I'm currently running -rc7). Effects disappears when downgrading to kvm-kmod-2.6.35.6. Will see if I can bisect later, but maybe someone already has an idea or wants to reproduce (just run something like "find /" on one text console and witch to another one - text fragments will remain on the screen on every few switches).
Quoted 4 lines
Reproduces on kvm.git. I wonder what's going on.
Looks like vesafb uses the bios to switch the display start, so I expect a problem in qemu reacting to this.
Hm, you said it is a kernel regression. Maybe it's an issue with dirty bit tracking.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 13 lines · Avi Kivity
Quoted 12 lines · Avi Kivity
Quoted 10 lines · Jan Kiszka
Hi,
I'm seeing quite frequent corruptions of the VESA frame buffer with Linux guests (vga=0x317) that are starting with KVM kernel modules of upcoming 2.6.36 (I'm currently running -rc7). Effects disappears when downgrading to kvm-kmod-2.6.35.6. Will see if I can bisect later, but maybe someone already has an idea or wants to reproduce (just run something like "find /" on one text console and witch to another one - text fragments will remain on the screen on every few switches).
Quoted 9 lines
Quoted 5 lines
Reproduces on kvm.git. I wonder what's going on.
Looks like vesafb uses the bios to switch the display start, so I expect a problem in qemu reacting to this.
Hm, you said it is a kernel regression. Maybe it's an issue with dirty bit tracking.
Ah, cross-posting...
It need not be a kernel thing, see my other mail.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux
Quoted 9 lines · Jan Kiszka
Hi,
I'm seeing quite frequent corruptions of the VESA frame buffer with Linux guests (vga=0x317) that are starting with KVM kernel modules of upcoming 2.6.36 (I'm currently running -rc7). Effects disappears when downgrading to kvm-kmod-2.6.35.6. Will see if I can bisect later, but maybe someone already has an idea or wants to reproduce (just run something like "find /" on one text console and witch to another one - text fragments will remain on the screen on every few switches).
Commit d25f31f488e5f7597c17a3ac7d82074de8138e3b in kvm.git ("KVM: x86:
avoid unnecessary bitmap allocation when memslot is clean") is at least magnifying the issue. With this patch applied, I can easily trigger display corruptions when switching between VGA consoles while one of them is undergoing heavy updates.
However, I once saw a much smaller inconsistency during my tests even with a previous revision. Maybe there is a fundamental issue in when and how the coalesced backlog is replayed, and this commit just makes the corruptions more likely. This may even be a QEMU issue in the cirrus/vga model (both qemu-kvm and upstream show the effect).
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux
Quoted 17 lines · Jan Kiszka
Quoted 9 lines · Jan Kiszka
Hi,
I'm seeing quite frequent corruptions of the VESA frame buffer with Linux guests (vga=0x317) that are starting with KVM kernel modules of upcoming 2.6.36 (I'm currently running -rc7). Effects disappears when downgrading to kvm-kmod-2.6.35.6. Will see if I can bisect later, but maybe someone already has an idea or wants to reproduce (just run something like "find /" on one text console and witch to another one - text fragments will remain on the screen on every few switches).
Commit
d25f31f488e5f7597c17a3ac7d82074de8138e3bin kvm.git ("KVM: x86:
avoid unnecessary bitmap allocation when memslot is clean") is at least magnifying the issue. With this patch applied, I can easily trigger display corruptions when switching between VGA consoles while one of them is undergoing heavy updates.
Quoted 3 lines
However, I once saw a much smaller inconsistency during my tests even with a previous revision. Maybe there is a fundamental issue in when and how the coalesced backlog is replayed,
I didn't see any mmio writes to the framebuffer, so I don't think coalescing plays a part here.
Quoted 4 lines
and this commit just makes the
corruptions more likely. This may even be a QEMU issue in the cirrus/vga model (both qemu-kvm and upstream show the effect).
What about -no-kvm?
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 8 lines · Avi Kivity
Quoted 5 lines
and this commit just makes the
corruptions more likely. This may even be a QEMU issue in the cirrus/vga model (both qemu-kvm and upstream show the effect).What about -no-kvm?
Doesn't happen there.
My guess is the initial dirty log after the switch does not contain all 1s so memory isn't updated. Probably need to force a full redraw in that case.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 25 lines · Avi Kivity
Quoted 17 lines · Jan Kiszka
Quoted 9 lines · Jan Kiszka
Hi,
I'm seeing quite frequent corruptions of the VESA frame buffer with Linux guests (vga=0x317) that are starting with KVM kernel modules of upcoming 2.6.36 (I'm currently running -rc7). Effects disappears when downgrading to kvm-kmod-2.6.35.6. Will see if I can bisect later, but maybe someone already has an idea or wants to reproduce (just run something like "find /" on one text console and witch to another one - text fragments will remain on the screen on every few switches).
Commit
d25f31f488e5f7597c17a3ac7d82074de8138e3bin kvm.git ("KVM: x86:
avoid unnecessary bitmap allocation when memslot is clean") is at least magnifying the issue. With this patch applied, I can easily trigger display corruptions when switching between VGA consoles while one of them is undergoing heavy updates.Quoted 3 lines
However, I once saw a much smaller inconsistency during my tests even with a previous revision. Maybe there is a fundamental issue in when and how the coalesced backlog is replayed,
I didn't see any mmio writes to the framebuffer, so I don't think coalescing plays a part here.
Quoted 6 lines
Quoted 4 lines
and this commit just makes the
corruptions more likely. This may even be a QEMU issue in the cirrus/vga model (both qemu-kvm and upstream show the effect).What about -no-kvm?
Just booted it (took ages), and the result was actually a completely black screen. Kind of persistent corruption. This really looks like a qemu issue now, maybe even a regression as I don't remember running into such effects a while back.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux
Quoted 6 lines · Jan Kiszka
Quoted 6 lines
Quoted 5 lines
and this commit just makes the
corruptions more likely. This may even be a QEMU issue in the cirrus/vga model (both qemu-kvm and upstream show the effect).
Quoted 6 lines
Quoted 1 line
What about -no-kvm?
Just booted it (took ages), and the result was actually a completely black screen. Kind of persistent corruption. This really looks like a qemu issue now, maybe even a regression as I don't remember running into such effects a while back.
Worked fine for me (though yes it was slow - did tcg regress?).
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 7 lines · Avi Kivity
Quoted 6 lines · Jan Kiszka
Quoted 6 lines
Quoted 5 lines
and this commit just makes the
corruptions more likely. This may even be a QEMU issue in the cirrus/vga model (both qemu-kvm and upstream show the effect).
Quoted 9 lines
Quoted 6 lines
Quoted 1 line
What about -no-kvm?
Just booted it (took ages), and the result was actually a completely black screen. Kind of persistent corruption. This really looks like a qemu issue now, maybe even a regression as I don't remember running into such effects a while back.
Worked fine for me (though yes it was slow - did tcg regress?).
I reread my commit but could not find any reason to make this corruption in kernel side.
But at least, I want to make it clear whether my commit was just a magnifier of this problem, I know that magnifier itself is bad, or not.
Though I'm now trying some debugging but have not got a way to show kvm_vm_ioctl_get_dirty_log() is 100% correct yet.
If you have any idea please tell me!
Takuya
Quoted 18 lines · Takuya Yoshikawa
Quoted 7 lines · Avi Kivity
Quoted 6 lines · Jan Kiszka
Quoted 6 lines
Quoted 5 lines
and this commit just makes the
corruptions more likely. This may even be a QEMU issue in the cirrus/vga model (both qemu-kvm and upstream show the effect).Quoted 9 lines
Quoted 6 lines
Quoted 1 line
What about -no-kvm?
Just booted it (took ages), and the result was actually a completely black screen. Kind of persistent corruption. This really looks like a qemu issue now, maybe even a regression as I don't remember running into such effects a while back.
Worked fine for me (though yes it was slow - did tcg regress?).
Quoted 9 lines
I reread my commit but could not find any reason to make this corruption in kernel side.
But at least, I want to make it clear whether my commit was just a magnifier of this problem, I know that magnifier itself is bad, or not.
Though I'm now trying some debugging but have not got a way to show
kvm_vm_ioctl_get_dirty_log()is 100% correct yet.
I tried some tests and found another issue.
I opened a terminal in one workspace and did "find /" on it. Then I switched to another workspace. When I switched back to the first one, the display of "find /" result was frozen.
I also replaced kvm_vm_ioctl_get_dirty_log() with the version before my patch was applied and got the same result.
I don't know this is related to your report, but something seems wrong.
Thanks,
Takuya
Quoted 31 lines · Takuya Yoshikawa
Quoted 18 lines · Takuya Yoshikawa
Quoted 7 lines · Avi Kivity
Quoted 6 lines · Jan Kiszka
Quoted 6 lines
Quoted 5 lines
and this commit just makes the
corruptions more likely. This may even be a QEMU issue in the cirrus/vga model (both qemu-kvm and upstream show the effect).Quoted 9 lines
Quoted 6 lines
Quoted 1 line
What about -no-kvm?
Just booted it (took ages), and the result was actually a completely black screen. Kind of persistent corruption. This really looks like a qemu issue now, maybe even a regression as I don't remember running into such effects a while back.
Worked fine for me (though yes it was slow - did tcg regress?).
Quoted 9 lines
I reread my commit but could not find any reason to make this corruption in kernel side.
But at least, I want to make it clear whether my commit was just a magnifier of this problem, I know that magnifier itself is bad, or not.
Though I'm now trying some debugging but have not got a way to show
kvm_vm_ioctl_get_dirty_log()is 100% correct yet.I tried some tests and found another issue.
Quoted 8 lines
I opened a terminal in one workspace and did "find /" on it. Then I switched to another workspace. When I switched back to the first one, the display of "find /" result was frozen.
I also replaced
kvm_vm_ioctl_get_dirty_log()with the version before my patch was applied and got the same result.I don't know this is related to your report, but something seems wrong.
Besides corruptions, I'm getting those freezes from time to time as well (fairly annoying now). Once that happened, (SDL) display updates can only be achieved by pressing CTRL-ATL-1 (ie. switching qemu consoles).
Thanks for looking into this. I also think your patch is not guilty. But we still need to understand the real issue.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux