Fix for DOS keyboard problems
PATCH 1/1 Fix for DOS keyboard problems
This is a repost as inline diff - there don't seem to be many attachments on this list.
Scratching an itch for myself, I have created a patch that allows me to use Turbo Pascal and other Turbo Vision-based programs from DOS. I post it here in the hope that someone take a good look, and maybe, there's even a chance of this going into mainline. I verified that it doesn't break keyboard or mouse functionality in Win 3.11, Win 2000 and Linux X11 (from systemrescuecd 1.1.4) as well as some DOS games. Couldn't get win98 to run, unfortunately, so no testing there. However, there is always the chance that it might break some random obscure stuff, and clearly I cannot test everything.
The patch basically exploits the fact that the BIOS IRQ 1 handler always does <disable keyboard> - <read 60h> - <enable keyboard> in this order. We just remember the last value read from port 60h outside this disable/enable block and play it back inside. The keyboard is still very unreliable when a keyboard driver is loaded inside DOS (I assume that the keyboard driver completely disables the BIOS handler). This behavior is also present without my patch, and I personally don't care about it, so this should not be an obstacle.
I also didn't know what to do about kbd_save and kbd_load just because I didn't dig into them to find out what they are used for. The new member should probably be serialized there.
My ultimate hope would be to get this into VirtualBox (see ticket #58 [1]). It should be easy enough to adapt the patch to VirtualBox; I will try to do that within the next few days.
PATCH 1/1 Fix for DOS keyboard problems
hw/pckbd.c +15 -4
@@ -120,6 +120,7 @@uint8_t mode;/* Bitmask of devices with data available. */ uint8_t pending; +uint32_t held_kbd_val;void *kbd;void *mouse;
@@ -161,9 +162,10 @@ {KBDState *s = (KBDState *)opaque; -if (level) +if (level) { s->pending |=KBD_PENDING_KBD ; -else + s->held_kbd_val = (uint32_t )-1 ; + }else s->pending &= ~KBD_PENDING_KBD ;kbd_update_irq (s); }
@@ -238,6 +240,7 @@break ;case KBD_CCMD_KBD_ENABLE : s->mode &= ~KBD_MODE_DISABLE_KBD ; + s->held_kbd_val = (uint32_t )-1 ;kbd_update_irq (s);break ;case KBD_CCMD_READ_INPORT :
@@ -283,8 +286,15 @@if (s->pending ==KBD_PENDING_AUX ) val =ps2_read_data (s->mouse); -else - val =ps2_read_data (s->kbd); +else { +if (s->mode &KBD_MODE_DISABLE_KBD && s->held_kbd_val != (uint32_t )-1 ) + val = s->held_kbd_val; +else { + val =ps2_read_data (s->kbd); +if (!(s->mode &KBD_MODE_DISABLE_KBD )) + s->held_kbd_val = val; + } + }#if defined(DEBUG_KBD )printf ("kbd: read data=0x%02x\n" , val);
@@ -339,6 +349,7 @@ s->mode =KBD_MODE_KBD_INT |KBD_MODE_MOUSE_INT ; s->status =KBD_STAT_CMD |KBD_STAT_UNLOCKED ; + s->held_kbd_val = (uint32_t )-1 ; }static void kbd_save (QEMUFile * f,void * opaque)
Quoted 5 lines · Stefan Ring
The keyboard is still very unreliable when a keyboard driver is loaded inside DOS (I assume that the keyboard driver completely disables the BIOS handler). This behavior is also present without my patch, and I personally don't care about it, so this should not be an obstacle.
Can you say a bit more about what your patch actually fixes?
I've been using QEMU with several versions of MS-DOS and haven't noticed any keyboard problems.
I even use various "halt-on-idle" idle programs, and haven't noticed any keyboard problems.
So what behaviour does your patch fix?
-- Jamie
Quoted 9 lines · Jamie Lokier
Quoted 5 lines · Stefan Ring
The keyboard is still very unreliable when a keyboard driver is loaded inside DOS (I assume that the keyboard driver completely disables the BIOS handler). This behavior is also present without my patch, and I personally don't care about it, so this should not be an obstacle.
Can you say a bit more about what your patch actually fixes?
Quoted 7 lines
I've been using QEMU with several versions of MS-DOS and haven't noticed any keyboard problems.
I even use various "halt-on-idle" idle programs, and haven't noticed any keyboard problems.
So what behaviour does your patch fix?
The particular problem that I noticed and that is 100% reproducible and also very understandable when you look at what the DOS programs do, is that in Borland's Text-Mode DOS IDEs (most likely BC++ 3.1, 4, Turbo Pascal 6 but definitely Turbo Pascal 7) and IIRC also in Turbo Vision programs generated by them, every cursor key press is interpreted twice. So when you open up the leftmost menu and press the down cursor key once, the third entry gets selected instead of the second one.
The reason for this is that said programs install an IRQ 1 handler which does little more than read from port 60h and pass control to the underlying BIOS (or keyboard driver) handler. The BIOS handler reads port 60h again, and should apparently see the same value as the first handler. The problem with the cursor keys is that they generate two scan codes in succession on port 60h that should be read by two separate IRQ handler activations but because QEMU's original behavior is to consume one data byte per read from 60h, the data gets used up too fast.
Quoted 9 lines · Stefan Ring
The particular problem that I noticed and that is 100% reproducible and also very understandable when you look at what the DOS programs do, is that in Borland's Text-Mode DOS IDEs (most likely BC++ 3.1, 4, Turbo Pascal 6 but definitely Turbo Pascal 7) and IIRC also in Turbo Vision programs generated by them, every cursor key press is interpreted twice. So when you open up the leftmost menu and press the down cursor key once, the third entry gets selected instead of the second one.
Quoted 9 lines
The reason for this is that said programs install an IRQ 1 handler which does little more than read from port 60h and pass control to the underlying BIOS (or keyboard driver) handler. The BIOS handler reads port 60h again, and should apparently see the same value as the first handler. The problem with the cursor keys is that they generate two scan codes in succession on port 60h that should be read by two separate IRQ handler activations but because QEMU's original behavior is to consume one data byte per read from 60h, the data gets used up too fast.
Let's see if I understand your explanation.
1. Cursor key is pressed. The key press is represented as two scan codes.
2. IRQ 1 is entered.
3. Borland's code reads port 60h - gets the first scan code.
4. BIOS's code reads port 60h - gets the second scan code.
5. Return from IRQ 1.
6. Cursor key is released. The key release is represented as two scan codes.
7. IRQ 1 is entered.
8. Borland's code reads port 60h - gets the first scan code.
9. BIOS's code reads port 60h - gets the second scan code.
10. Return from IRQ 1.
So both Borland's code and the BIOS are *missing* scan codes.
How does that result in Borland seeing *multiple* cursor key press/release sequences?
Apparently when port 0x60 is read, that de-asserts IRQ1, resets the IBF flag ("input buffer full"), and another byte could be received from the keyboard. However, reading port 0x60 quickly, before another byte can be received over the keyboard cable, should return the same byte.
So I agree that the emulated port 0x60 should return the same value if there has not been enough time for the (emulated) keyboard cable to transmit another scan code.
But detecting the particular sequence used by Borland code and the BIOS together is a hack. I'm not surprised that, as you say, other DOS keyboard drivers remain broken after the patch.
If it goes in, the patch should include a very clear comment that the "held" value and detecting the disable/read/enable sequence is only a workaround for what Borland does and also depends on the BIOS sequence, and is not a correct emulation in general.
-- Jamie
Quoted 15 lines · Jamie Lokier
Let's see if I understand your explanation.
1. Cursor key is pressed. The key press is represented as two scan codes.
2. IRQ 1 is entered.
3. Borland's code reads port 60h - gets the first scan code.
4. BIOS's code reads port 60h - gets the second scan code.
5. Return from IRQ 1.6. Cursor key is released. The key release is represented as two scan codes.
7. IRQ 1 is entered.
8. Borland's code reads port 60h - gets the first scan code.
9. BIOS's code reads port 60h - gets the second scan code.
10. Return from IRQ 1.
Quoted 4 lines
So both Borland's code and the BIOS are *missing* scan codes.
How does that result in Borland seeing *multiple* cursor key press/release sequences?
Actually I wondered about the same thing while I wrote my previous answer... I'm not exactly sure, but it seems to happen like this: the cursor keys generate two scan codes - E0h and the real scan code in this order. Somehow the second IRQ seems to happen although its value has already been consumed by the first interrupt handler activation, and during the second invocation the handler will just get repetitions of the last value read. Apparently the scan code for the cursor keys is interpreted as cursor movement regardless if E0h appeared earlier or not.
Another symptom is that when typing text inside Turbo Pascal, about 1/3 - 1/4 of the characters typed are doubled randomly.
Quoted 2 lines
But detecting the particular sequence used by Borland code and the BIOS together is a hack.
Yes, it is a hack. But it's not only Borland - there are some reports about some other software in various places as well. The VirtualBox cases [1] and [2] mention some more, and then there is a post in a german discussion board[3] (the linked post is written in english) about this strange "OS"[4]. The german one is interesting, although I'm not sure if his observations are correct. There is another one a few posts further down which states that the problem had already been discussed on the FreeDOS mailing list and is known to "the QEMU developers". Well, it must have been known to Fabrice Bellard when he first created it.
Quoted 2 lines
I'm not surprised that, as you say, other DOS keyboard drivers remain broken after the patch.
Apparently, DOS keyboard drivers don't normally lock and unlock the keyboard like the Bochs BIOS does.
Quoted 4 lines
If it goes in, the patch should include a very clear comment that the "held" value and detecting the disable/read/enable sequence is only a workaround for what Borland does and also depends on the BIOS sequence, and is not a correct emulation in general.
I agree, and there's already an "XXX" in that file stating that there should be a timer[5], although I don't completely understand that comment - I don't see how that would help because this function gets called every time the keyboard is re-enabled by the BIOS handler. Bochs does the timer thing correctly. I don't know how complicated it would be to add the timer to QEMU, but I expect it would have been done already if it were easy.The hack at least improves matters a bit.
[1] http://www.virtualbox.org/ticket/58
[2] http://www.virtualbox.org/ticket/1599
[3] http://dwforum.wcx-network.com/viewtopic.php?p=2266#2266
[4] http://www.deskwork.de/FOTOS/
[5] http://git.savannah.gnu.org/cgit/qemu.git/tree/hw/pckbd.c?id=2a1704a71d851624175509a2b15a000dcc7c0386#n132
Stefan Ring wrote:
...
Quoted 3 lines
Another symptom is that when typing text inside Turbo Pascal, about 1/3 - 1/4 of the characters typed are doubled randomly.
I sometimes see the same behavior when using starting an rdp session to windows servers from within a rdp session to local qemu-kvm windows xp guest. logging in to the server with long passwords is difficult.
Signature · 1 line
Hans
Quoted 21 lines · Stefan Ring
Quoted 15 lines · Jamie Lokier
Let's see if I understand your explanation.
1. Cursor key is pressed. The key press is represented as two scan codes.
2. IRQ 1 is entered.
3. Borland's code reads port 60h - gets the first scan code.
4. BIOS's code reads port 60h - gets the second scan code.
5. Return from IRQ 1.6. Cursor key is released. The key release is represented as two scan codes.
7. IRQ 1 is entered.
8. Borland's code reads port 60h - gets the first scan code.
9. BIOS's code reads port 60h - gets the second scan code.
10. Return from IRQ 1.Quoted 4 lines
So both Borland's code and the BIOS are *missing* scan codes.
How does that result in Borland seeing *multiple* cursor key press/release sequences?
Quoted 9 lines
Actually I wondered about the same thing while I wrote my previous answer... I'm not exactly sure, but it seems to happen like this: the cursor keys generate two scan codes - E0h and the real scan code in this order. Somehow the second IRQ seems to happen although its value has already been consumed by the first interrupt handler activation, and during the second invocation the handler will just get repetitions of the last value read. Apparently the scan code for the cursor keys is interpreted as cursor movement regardless if E0h appeared earlier or not.
[..snip..]
There's other DOS application that exhibits erroneous keyboard beahviour and i was curious whether your hack "fixes" it too (alas it doesn't) - Command and Conquer, the demo can be found at:
http://www.dosgamesarchive.com/download/command-and-conquer/
Keyboard is dysfunctional in both the setup and the game. What i'm wondering now if this is a indeed pckbd or BIOS problem, but Bochs where the former can be verified refuses to work on my machines, perhaps someone could give it a try and report here?
Signature · 1 line
mailto:[email protected]
2009/8/24 malc <[email protected]>:
Quoted 10 lines
There's other DOS application that exhibits erroneous keyboard beahviour and i was curious whether your hack "fixes" it too (alas it doesn't) - Command and Conquer, the demo can be found at:
http://www.dosgamesarchive.com/download/command-and-conquer/
Keyboard is dysfunctional in both the setup and the game. What i'm wondering now if this is a indeed pckbd or BIOS problem, but Bochs where the former can be verified refuses to work on my machines, perhaps someone could give it a try and report here?
I will try to run C&C in Bochs, and I'm definitely not surprised that my hack doesn't fix that as I'm almost 100% certain that it does its own keyboard processing and bypasses the BIOS entirely (as virtually all games do).
Quoted 12 lines · Stefan Ring
2009/8/24 malc <[email protected]>:
Quoted 10 lines
There's other DOS application that exhibits erroneous keyboard beahviour and i was curious whether your hack "fixes" it too (alas it doesn't) - Command and Conquer, the demo can be found at:
http://www.dosgamesarchive.com/download/command-and-conquer/
Keyboard is dysfunctional in both the setup and the game. What i'm wondering now if this is a indeed pckbd or BIOS problem, but Bochs where the former can be verified refuses to work on my machines, perhaps someone could give it a try and report here?
Quoted 4 lines
I will try to run C&C in Bochs, and I'm definitely not surprised that my hack doesn't fix that as I'm almost 100% certain that it does its own keyboard processing and bypasses the BIOS entirely (as virtually all games do).
I certainly saw the disable/enable sequence while trying to debug the issue.
Signature · 1 line
mailto:[email protected]
Quoted 4 lines · Stefan Ring
I will try to run C&C in Bochs, and I'm definitely not surprised that my hack doesn't fix that as I'm almost 100% certain that it does its own keyboard processing and bypasses the BIOS entirely (as virtually all games do).
On the other hand, I'd expect virtually all of the ones which handle the keyboard themselves (and disable the BIOS IRQ) to read port 0x60 exactly once per scan code.
Which implies something else is broken too.
Btw, a timer won't necessarily fix the problem seen with Borland, if QEMU/KVM are preempted at the wrong moment - unless there's a virtual CPU timer available which is not affected by host scheduling.
-- Jamie
Quoted 9 lines · Hans de Bruin
Stefan Ring wrote:
...Quoted 3 lines
Another symptom is that when typing text inside Turbo Pascal, about 1/3 - 1/4 of the characters typed are doubled randomly.
I sometimes see the same behavior when using starting an rdp session to windows servers from within a rdp session to local qemu-kvm windows xp guest. logging in to the server with long passwords is difficult.
That's likely to be a different problem, as RDP does not involve the emulated keyboard hardware at all, and is handled by Windows networking.
Unless you meant VNC for the outer connection?
-- Jamie
Quoted 3 lines · Jamie Lokier
Btw, a timer won't necessarily fix the problem seen with Borland, if QEMU/KVM are preempted at the wrong moment - unless there's a virtual CPU timer available which is not affected by host scheduling.
That's what I thought. Bochs uses a virtual CPU timer but that's easy to do in Bochs because everything is emulated. I suppose the same could be done with QEMU for 16 bit mode because everything is emulated as well, AFAIK. The more native 32 bit modes would be much more difficult or almost impossible, though. And it would not be good to have port 60h behave differently in 16 bit mode vs. 32 bit mode.
Just some random thoughts...
Quoted 5 lines · Jamie Lokier
On the other hand, I'd expect virtually all of the ones which handle the keyboard themselves (and disable the BIOS IRQ) to read port
0x60exactly once per scan code.Which implies something else is broken too.
True. I would not be too surprised if the aforementioned XXX-commented hack (no interrupt when kbd disabled) had something to do with this.
Quoted 12 lines · Jamie Lokier
Quoted 5 lines · Hans de Bruin
Stefan Ring wrote:
...Quoted 2 lines
Another symptom is that when typing text inside Turbo Pascal, about 1/3 - 1/4 of the characters typed are doubled randomly.
I sometimes see the same behavior when using starting an rdp session to windows servers from within a rdp session to local qemu-kvm windows xp guest. logging in to the server with long passwords is difficult.
That's likely to be a different problem, as RDP does not involve the emulated keyboard hardware at all, and is handled by Windows networking.
Unless you meant VNC for the outer connection?
sorry, you are right, the keyboard there is no keyboard involved here.
Signature · 1 line
Hans
Quoted 1 line
There's a hard disk image for VirtualBox attached to this bug report. I just converted it to RAW format ("VBoxManage clonehd") and ran it with QEMU. The behavior in QEMU is exactly the same as in VirtualBox. With my patch applied, the double cursor symptom disappears in all cases, even with the Russian keyboard driver. There is another problem, though, which is not affected by the presence or absence of my patch at all. Keyboard and mouse seem to influence and severely disturb each other when used simultaneously. This behavior is also present in VirtualBox.
Quoted 11 lines · Stefan Ring
Quoted 1 line
There's a hard disk image for VirtualBox attached to this bug report. I just converted it to RAW format ("VBoxManage clonehd") and ran it with QEMU. The behavior in QEMU is exactly the same as in VirtualBox. With my patch applied, the double cursor symptom disappears in all cases, even with the Russian keyboard driver. There is another problem, though, which is not affected by the presence or absence of my patch at all. Keyboard and mouse seem to influence and severely disturb each other when used simultaneously. This behavior is also present in VirtualBox.
Damn, it's actually the other VirtualBox ticket:
http://www.virtualbox.org/ticket/58
Quoted 10 lines
There's other DOS application that exhibits erroneous keyboard beahviour and i was curious whether your hack "fixes" it too (alas it doesn't) - Command and Conquer, the demo can be found at:
http://www.dosgamesarchive.com/download/command-and-conquer/
Keyboard is dysfunctional in both the setup and the game. What i'm wondering now if this is a indeed pckbd or BIOS problem, but Bochs where the former can be verified refuses to work on my machines, perhaps someone could give it a try and report here?
I just tried to run C&C on QEMU, VirtualBox and Bochs. Unfortunately, the game itself crashed immediately during startup, but the setup program worked. It also accepts keyboard input for specifying sound hardware parameters. The results:
QEMU: mouse works, keyboard does not, keyboard input is buffered and played back at the command prompt after program exits.
VirtualBox: the same
Bochs: neither keyboard nor mouse work
Quoted 11 lines · Stefan Ring
Quoted 10 lines
There's other DOS application that exhibits erroneous keyboard beahviour and i was curious whether your hack "fixes" it too (alas it doesn't) - Command and Conquer, the demo can be found at:
http://www.dosgamesarchive.com/download/command-and-conquer/
Keyboard is dysfunctional in both the setup and the game. What i'm wondering now if this is a indeed pckbd or BIOS problem, but Bochs where the former can be verified refuses to work on my machines, perhaps someone could give it a try and report here?
Quoted 4 lines
I just tried to run C&C on QEMU, VirtualBox and Bochs. Unfortunately, the game itself crashed immediately during startup, but the setup program worked. It also accepts keyboard input for specifying sound hardware parameters. The results:
I'm confused. Should the above be interpreted as: keyboard works in setup under Bochs, or?
Quoted 6 lines
QEMU: mouse works, keyboard does not, keyboard input is buffered and played back at the command prompt after program exits.
VirtualBox: the same
Bochs: neither keyboard nor mouse work
Signature · 1 line
mailto:[email protected]
Quoted 13 lines · malc
Quoted 11 lines · Stefan Ring
Quoted 10 lines
There's other DOS application that exhibits erroneous keyboard beahviour and i was curious whether your hack "fixes" it too (alas it doesn't) - Command and Conquer, the demo can be found at:
http://www.dosgamesarchive.com/download/command-and-conquer/
Keyboard is dysfunctional in both the setup and the game. What i'm wondering now if this is a indeed pckbd or BIOS problem, but Bochs where the former can be verified refuses to work on my machines, perhaps someone could give it a try and report here?
Quoted 7 lines
Quoted 4 lines
I just tried to run C&C on QEMU, VirtualBox and Bochs. Unfortunately, the game itself crashed immediately during startup, but the setup program worked. It also accepts keyboard input for specifying sound hardware parameters. The results:
I'm confused. Should the above be interpreted as: keyboard works in setup under Bochs, or?
No. I just meant that there is a way to test keyboard input in the setup program because it would accept keyboard input if it worked. It does not work in any of the three.
Quoted 19 lines · Stefan Ring
Quoted 18 lines · malc
Quoted 15 lines · Stefan Ring
Quoted 10 lines
There's other DOS application that exhibits erroneous keyboard beahviour and i was curious whether your hack "fixes" it too (alas it doesn't) - Command and Conquer, the demo can be found at:
http://www.dosgamesarchive.com/download/command-and-conquer/
Keyboard is dysfunctional in both the setup and the game. What i'm wondering now if this is a indeed pckbd or BIOS problem, but Bochs where the former can be verified refuses to work on my machines, perhaps someone could give it a try and report here?
I just tried to run C&C on QEMU, VirtualBox and Bochs. Unfortunately, the game itself crashed immediately during startup, but the setup program worked. It also accepts keyboard input for specifying sound hardware parameters. The results:
Quoted 6 lines
Quoted 2 lines
I'm confused. Should the above be interpreted as: keyboard works in setup under Bochs, or?
No. I just meant that there is a way to test keyboard input in the setup program because it would accept keyboard input if it worked. It does not work in any of the three.
I just can not parse the second sentence, but first one provides nice summary.
Signature · 1 line
mailto:[email protected]
Quoted 19 lines · Stefan Ring
Quoted 18 lines · malc
Quoted 15 lines · Stefan Ring
Quoted 10 lines
There's other DOS application that exhibits erroneous keyboard beahviour and i was curious whether your hack "fixes" it too (alas it doesn't) - Command and Conquer, the demo can be found at:
http://www.dosgamesarchive.com/download/command-and-conquer/
Keyboard is dysfunctional in both the setup and the game. What i'm wondering now if this is a indeed pckbd or BIOS problem, but Bochs where the former can be verified refuses to work on my machines, perhaps someone could give it a try and report here?
I just tried to run C&C on QEMU, VirtualBox and Bochs. Unfortunately, the game itself crashed immediately during startup, but the setup program worked. It also accepts keyboard input for specifying sound hardware parameters. The results:
Quoted 6 lines
Quoted 2 lines
I'm confused. Should the above be interpreted as: keyboard works in setup under Bochs, or?
No. I just meant that there is a way to test keyboard input in the setup program because it would accept keyboard input if it worked. It does not work in any of the three.
Red herring, but good enough excuse to brush up my long forgotten re-ing skils, C&C's code does this:
cseg01:0003550B mov bl, 60h cseg01:0003550D mov bh, 6 cseg01:0003550F mov eax, 200h cseg01:00035514 cseg01:00035514 loc_35514: ; CODE XREF: sub_35410+115j cseg01:00035514 int 31h ; DPMI Services ax=func xxxxh cseg01:00035514 ; GET REAL MODE INTERRUPT VECTOR cseg01:00035514 ; BL = interrupt number cseg01:00035514 ; Return: CF clear, CX:DX = segment:offset of real mode interrupt handler cseg01:00035516 jb loc_356DC cseg01:0003551C or cx, dx cseg01:0003551F jz short loc_3552C cseg01:00035521 inc bl cseg01:00035523 dec bh cseg01:00035525 jnz short loc_35514 cseg01:00035527 jmp loc_356DC
Basically iterating over interrupts 0x60..0x66 looking for an empty slot if none was found it basically skips the whole installation of keyboard handling routine and hence keyboard doesn't work.
Connecting gdbstub and doing "set *(long*)384 = 0" inside gdb prior to running C&C's setup, made keyboard "functional".
So BIOS and some interesting expectations of C&C are to blame here not pckbd.c.
Signature · 1 line
mailto:[email protected]