can I make this work… (Foundation for accessibility project)

this is a rather different use case than what you've been thinking of for KVM. It could mean significant improvement of the quality of life of disabled programs like myself. It's difficult to convey what it's like to try to use computers with speech recognition for something other than writing so, bear with me when I say something is real but don't quite prove it yet. also, please take it as read that the only really usable speech recognition environment out there is NaturallySpeaking with Google close behind in terms of accuracy but not even in the same planet for ability to extend for speech enabled applications.

I'm trying to figure out ways of making it possible to drive Linux from Windows speech recognition (NaturallySpeaking). The goal is a system where Windows runs in a virtual machine (Linux host), audio is passed through from a USB headset to the Windows environment. And the output of the recognition engine is piped through some magic back to the Linux host.

the hardest part of all of this without question is getting clean uninterrupted audio from the USB device all the way through to the Windows virtual machine. virtual box, VMware fail mostly in delivering reliable audio to the virtual machine.

I expect KVM to not work right with regards to getting clean audio/real-time USB but I'm asking in case I'm wrong. if it doesn't work or can't work yet, what would it take to make it possible for clean audio to be passed through to a guest?

--- Why this is important, approaches that failed, why think this will work. Boring accessibility info ---

The history of trying to make Windows or DOS based speech recognition drive Linux has a long and tortured history. almost all of them involve some form of an open loop system that ignores system context and counts on the grammar to specify the context and the subsequent keystrokes injected into the target system.

This model fails because it effectively speaking keyboard functions which wastes the majority of the power of a good grammar in a speech recognition environment.

Most common configuration for speech recognition in a virtualized environment today is that Windows is the host with speech recognition and Linux is the guest. It's just a reimplementation of the open-loop system described above where your dictation results are keystrokes injected into the virtual machine console window. Sometimes works, sometimes drops characters.

One big failing of the Windows host/Linux guest environments is in addition to dropping characters,it seems to drop segments of the audio stream on the Windows side. It's common but not frequent for this to happen anyway when running Windows with any sort of CPU utilization but it's almost guaranteed as soon as a virtual machine starts up.

Another failing is that the context the recognition application is aware of is the window of the console. It knows nothing about the internal context of the virtual machine (what application has focus). And unfortunately it can't know anything more because of the way that NaturallySpeaking uses the local Windows context.

Inverting the relationship between guest and host where Linux is the host and Windows is the guest solves at least the focus problem. In the virtual machine, you have a portal application the canal control the perception of context and tunnels the character stream from the recognition engine into the host OS to drive it open loop. The portal application[1] can also communicate which grammar sequence has been parsed and what action should be taken on the host site. At this point, we now have the capabilities of a closed-loop speech recognition environment where a grammar can read context to generate a new grammar to fit the applications state. This means smaller utterances which can be disambiguated versus the more traditional large utterance disambiguation technique.

A couple other advantages of Windows as a guest is that it only run speech recognition in the portal. There's no browsers, no flash, JavaScript, viruses and other "stuff" taking up resources and distracting from speech recognition working as well as possible. The downside is that the host running the virtual machine needs to make the VM very high almost real-time priority[2] so that it doesn't stall and speech recognition works as quickly and as accurately as possible.

Hope I didn't bore you too badly. Thank you for reading and I hope we can make this work. --- eric

[1] should I call it cake?
[2] I'm looking at you Firefox, sucking down 30% of the CPU doing nothing

I'm adding two people who might know.

Do you have any idea what the "magic to pipe data back to the Linux host" should look like? Does a normal serial port (COM1 for Windows, /dev/ttyS0 for Linux) work?

Paolo

Hi,

kvm's usb pass-through should be able to handle this without any issues (other then some latency), it uses special buffering for isochronous usb packets, which should take care of usb audio working.

I've never tested audio recording, but audio playback and video recording (webcams) work, so I expect audio recording to be fine.

Regards,

Hans

That's great to know. I will spin up a version of Windows 7 and give it a try given that I'm not looking at it, I can strip it down to the barest user interface elements and improve performance significantly.

FYI, Windows 8 actually works better for hand crips like myself because it's easier to fit the big blocky icons with hand tremors and has flaky fine motion control them is to try and hit the tiny little icons and menus. Yes, that's why I like unity as well. :-) I would love to have a capability to make the title bar drop downs bigger temporarily so I could more easily pick elements out of them but otherwise have been visually out-of-the-way.

I tried it and it took me approximately 10 to 12 hours to install Windows 7 twice and I didn't even finish installing the last time.

Here's what happened. The first time I installed it, it was a naïve install. Took all the defaults just set up the ISO and let the install run. Then I installed all the updates. Hours went by and it kind of came up and ran but then I tried to install the virt I/O drivers and the Windows installation lost its mind. Did some reading on how to make performance better and on using the virtio drivers in windows.

So I start of the second install, same size disk 25 GB, same amount of RAM, 1 GB and installed the ethernet, disk and balloon drivers at the right time. I also changed the cache to none, I/O something to native and I think that's about it. Anyway, that was not really any improvement. It's still was incredibly slow and this time it was complaining about running out of memory and packages install never finished. Just kept going and going going. iptraf reported network io ranging from 3kbit to 100kbit range when the updates were running.

I'm accustomed to lesser performance on virtual machines. That's the hazard of a running on old and slow laptop (dell e6400 (2.2ghz core duo, 8gb ram)[1]) but even virtual box is not this slow. So what am I doing wrong? It would be nice to use a slow machine like this as many handcrips don't have a whole lot of resources for buying newer/faster machines. On the other hand, many of them use desktops and work from one place whereas someone like me is all over the map (quite literally).

--- eric
[1] Part of the reason I don't bother upgrading machines all that often is because it no matter how fast the CPU runs or how much memory I have, Windows always runs about the same speed.

How did you start the virtual machine? Perhaps you're not using KVM but emulation? I have a fast machine but slow disk (a NAS on 100 MBit ethernet) and I can do about 15 automated installations in less than 6 hours.

Are you using libvirt or directly invoking QEMU?

Paolo

I was using one of the GUIs ( less hand stress than trying to assemble a commandline). Unfortunately I'm in Windows 8 right now because I'm writing. I'm very sure the GUI was http://virt-manager.org/ I tried a different one but it kept telling me I only had QEMU I thought "silly program, that can't be right". Someday I will not argue with software or small electronic boxes. They don't care who wins and they are much more stubborn than I am.

I'll be able to run some tests in about 2 to 3 hours after I finish this document. Let me know what I should look at? on a side note, a pointer to an automated install process would be wonderful.

GNOME Boxes can pretty much automate the install process.

Can you just run "ps aux" while the install is running and send the result?

Paolo

I went back and verified I had installed all packages. apparently I missed a few updates. also I was more familiar with the UI tool. I noticed a few places where kvm was now an option. last I made a copy of the dvd to an iso as an install image. end result is *wow* much faster. I now have hope that my project will work. sure does like giving 110% in cpu speed.

4384 libvirt+ 20 0 2825112 2.058g 9960 R 109.1 26.6 12:47.73 qemu-system-x86

next report after updates install

btw, would you like a better UI design for a management tool? I have some ideas but would need someone with hands to put it together.

--- eric

top sez

Tasks: 182 total, 4 running, 178 sleeping, 0 stopped, 0 zombie
%Cpu(s): 44.2 us, 14.9 sy, 0.0 ni, 38.7 id, 2.0 wa, 0.0 hi, 0.2 si, 0.0 st

KiB Mem:   8128204 total,  4750320 used,  3377884 free,    54476 buffers
KiB Swap:  8338428 total,        0 used,  8338428 free.  1996164 cached Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
COMMAND
4384 libvirt+ 20 0 2634992 2.033g 9940 R 108.6 26.2 2:02.83
qemu-syste+

 2668 eric      20   0 1284184  66308  29828 S   2.3  0.8   0:21.50 compiz
 1314 root      20   0 1032288  22264  11436 S   2.0  0.3   0:46.29 
libvirtd
   18 root      20   0       0      0      0 S   1.7  0.0   0:00.96 
kworker/1:0
 1423 root      20   0  410736  49196  35228 S   1.7  0.6   0:32.18 Xorg
 4694 root      20   0       0      0      0 R   1.7  0.0   0:00.20 
kworker/0:1
 2837 eric      20   0 1481612 102828  38476 S   1.0  1.3   0:54.03 python
 2628 eric      20   0   20232    940    768 S   0.3  0.0   0:00.69 

syndaemon
3047 eric 20 0 653160 20868 12472 S 0.3 0.3 0:02.14
gnome-term+
3147 eric 20 0 377868 4168 3288 S 0.3 0.1 0:00.04
deja-dup-m+

    1 root      20   0   33908   3280   1472 S   0.0  0.0   0:01.62 init
    2 root      20   0       0      0      0 S   0.0  0.0   0:00.00 
kthreadd
    3 root      20   0       0      0      0 S   0.0  0.0   0:00.16 
ksoftirqd/0
    4 root      20   0       0      0      0 S   0.0  0.0   0:00.72 
kworker/0:0
    5 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 
kworker/0:+
    7 root      20   0       0      0      0 S   0.0  0.0   0:00.50 
rcu_sched
    8 root      20   0       0      0      0 R   0.0  0.0   0:00.40 

rcuos/0
eric@garnet:~$
ps aux sez

eric@garnet:~$ ps -aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.1  0.0  33908  3280 ?        Ss   11:12   0:01 /sbin/init
root         2  0.0  0.0      0     0 ?        S    11:12   0:00 [kthreadd]
root         3  0.0  0.0      0     0 ?        S    11:12   0:00 [ksoftirqd/0]
root         4  0.0  0.0      0     0 ?        S    11:12   0:00 [kworker/0:0]
root         5  0.0  0.0      0     0 ?        S<   11:12   0:00 [kworker/0:0H]
root         7  0.0  0.0      0     0 ?        S    11:12   0:00 [rcu_sched]
root         8  0.0  0.0      0     0 ?        S    11:12   0:00 [rcuos/0]
root         9  0.0  0.0      0     0 ?        S    11:12   0:00 [rcuos/1]
root        10  0.0  0.0      0     0 ?        S    11:12   0:00 [rcu_bh]
root        11  0.0  0.0      0     0 ?        S    11:12   0:00 [rcuob/0]
root        12  0.0  0.0      0     0 ?        S    11:12   0:00 [rcuob/1]
root        13  0.0  0.0      0     0 ?        S    11:12   0:00 [migration/0]
root        14  0.0  0.0      0     0 ?        S    11:12   0:00 [watchdog/0]
root        15  0.0  0.0      0     0 ?        S    11:12   0:00 [watchdog/1]
root        16  0.0  0.0      0     0 ?        S    11:12   0:00 [migration/1]
root        17  0.0  0.0      0     0 ?        S    11:12   0:00 [ksoftirqd/1]
root        18  0.0  0.0      0     0 ?        S    11:12   0:01 [kworker/1:0]
root        19  0.0  0.0      0     0 ?        S<   11:12   0:00 [kworker/1:0H]
root        20  0.0  0.0      0     0 ?        S<   11:12   0:00 [khelper]
root        21  0.0  0.0      0     0 ?        S    11:12   0:00 [kdevtmpfs]
root        22  0.0  0.0      0     0 ?        S<   11:12   0:00 [netns]
root        23  0.0  0.0      0     0 ?        S<   11:12   0:00 [writeback]
root        24  0.0  0.0      0     0 ?        S<   11:12   0:00 [kintegrityd]
root        25  0.0  0.0      0     0 ?        S<   11:12   0:00 [bioset]
root        26  0.0  0.0      0     0 ?        S<   11:12   0:00 [kworker/u5:0]
root        27  0.0  0.0      0     0 ?        S<   11:12   0:00 [kblockd]
root        28  0.0  0.0      0     0 ?        S<   11:12   0:00 [ata_sff]
root        29  0.0  0.0      0     0 ?        S    11:12   0:00 [khubd]
root        30  0.0  0.0      0     0 ?        S<   11:12   0:00 [md]
root        31  0.0  0.0      0     0 ?        S<   11:12   0:00 [devfreq_wq]
root        34  0.0  0.0      0     0 ?        S    11:12   0:00 [khungtaskd]
root        35  0.0  0.0      0     0 ?        S    11:12   0:00 [kswapd0]
root        36  0.1  0.0      0     0 ?        SN   11:12   0:02 [ksmd]
root        37  0.0  0.0      0     0 ?        SN   11:12   0:00 [khugepaged]
root        38  0.0  0.0      0     0 ?        S    11:12   0:00 [fsnotify_mark]
root        39  0.0  0.0      0     0 ?        S    11:12   0:00 [ecryptfs-kthrea]
root        40  0.0  0.0      0     0 ?        S<   11:12   0:00 [crypto]
root        52  0.0  0.0      0     0 ?        S<   11:12   0:00 [kthrotld]
root        74  0.0  0.0      0     0 ?        S<   11:12   0:00 [deferwq]
root        75  0.0  0.0      0     0 ?        S<   11:12   0:00 [charger_manager]
root       123  0.0  0.0      0     0 ?        S<   11:12   0:00 [kpsmoused]
root       124  0.0  0.0      0     0 ?        S<   11:12   0:00 [firewire]
root       125  0.0  0.0      0     0 ?        S<   11:12   0:00 [firewire_ohci]
root       126  0.0  0.0      0     0 ?        S    11:12   0:00 [scsi_eh_0]
root       127  0.0  0.0      0     0 ?        S    11:12   0:00 [scsi_eh_1]
root       128  0.0  0.0      0     0 ?        S    11:12   0:00 [scsi_eh_2]
root       129  0.0  0.0      0     0 ?        S    11:12   0:00 [scsi_eh_3]
root       130  0.0  0.0      0     0 ?        S    11:12   0:00 [scsi_eh_4]
root       131  0.0  0.0      0     0 ?        S    11:12   0:00 [scsi_eh_5]
root       133  0.0  0.0      0     0 ?        S    11:12   0:00 [kworker/u4:4]
root       139  0.0  0.0      0     0 ?        S    11:12   0:00 [scsi_eh_6]
root       140  0.0  0.0      0     0 ?        S    11:12   0:00 [usb-storage]
root       150  0.0  0.0      0     0 ?        S    11:12   0:00 [jbd2/sda3-8]
root       151  0.0  0.0      0     0 ?        S<   11:12   0:00 [ext4-rsv-conver]
root       317  0.0  0.0  19608   912 ?        S    11:12   0:00 upstart-udev-bridge --daemon
root       343  0.0  0.0  51752  1972 ?        Ss   11:12   0:01 /lib/systemd/systemd-udevd --daemon
root       369  0.0  0.0      0     0 ?        S    11:12   0:00 [jbd2/sdb3-8]
root       370  0.0  0.0      0     0 ?        S<   11:12   0:00 [ext4-rsv-conver]
root       426  0.0  0.0  15276   628 ?        S    11:12   0:00 upstart-file-bridge --daemon
root       471  0.0  0.0      0     0 ?        S<   11:12   0:00 [cfg80211]
syslog     506  0.0  0.0 255844  1276 ?        Ssl  11:12   0:00 rsyslogd
root       529  0.0  0.0      0     0 ?        S    11:12   0:00 [pccardd]
root       587  0.0  0.0      0     0 ?        S<   11:12   0:00 [kvm-irqfd-clean]
root       594  0.1  0.0      0     0 ?        S    11:12   0:01 [irq/46-iwlwifi]
root       603  0.0  0.0      0     0 ?        S<   11:12   0:00 [kmpathd]
root       604  0.0  0.0      0     0 ?        S<   11:12   0:00 [kmpath_handlerd]
root       647  0.0  0.0      0     0 ?        S<   11:12   0:00 [iwlwifi]
message+   655  0.0  0.0  40196  2480 ?        Ss   11:12   0:00 dbus-daemon --system --fork
root       679  0.0  0.0      0     0 ?        S<   11:12   0:00 [hd-audio0]
root       774  0.0  0.0  15260   396 ?        S    11:12   0:00 upstart-socket-bridge --daemon
root       776  0.0  0.0  19292  1712 ?        Ss   11:12   0:00 /usr/sbin/bluetoothd
root       819  0.0  0.0 330236  4432 ?        Ssl  11:12   0:00 /usr/sbin/ModemManager
root       831  0.0  0.0  43560  1924 ?        Ss   11:12   0:00 /lib/systemd/systemd-logind
root       851  0.0  0.0      0     0 ?        S<   11:12   0:00 [krfcommd]
avahi      872  0.0  0.0  32356  1668 ?        S    11:12   0:00 avahi-daemon: running [garnet.local]
avahi      874  0.0  0.0  32224   464 ?        S    11:12   0:00 avahi-daemon: chroot helper
root       937  0.0  0.1 345036  8612 ?        Ssl  11:12   0:00 NetworkManager
root       943  0.0  0.0 281472  5420 ?        Sl   11:12   0:00 /usr/lib/policykit-1/polkitd --no-debug
root      1065  0.0  0.0  20016   968 tty4     Ss+  11:12   0:00 /sbin/getty -8 38400 tty4
root      1069  0.0  0.0  20016   968 tty5     Ss+  11:12   0:00 /sbin/getty -8 38400 tty5
root      1071  0.0  0.0  30608  2652 ?        Ss   11:12   0:00 /sbin/wpa_supplicant -B -P /run/sendsigs.omit.d/w
root      1079  0.0  0.0  20016   968 tty2     Ss+  11:12   0:00 /sbin/getty -8 38400 tty2
root      1082  0.0  0.0  20016   968 tty3     Ss+  11:12   0:00 /sbin/getty -8 38400 tty3
root      1086  0.0  0.0  20016   960 tty6     Ss+  11:12   0:00 /sbin/getty -8 38400 tty6
kernoops  1171  0.0  0.0  37144  1008 ?        Ss   11:12   0:00 /usr/sbin/kerneloops
root      1178  0.0  0.0  75484  3404 ?        Ss   11:12   0:00 /usr/sbin/cups-browsed
root      1209  0.0  0.0   4368   692 ?        Ss   11:12   0:00 acpid -c /etc/acpi/events -s /var/run/acpid.socke
root      1247  0.0  0.0  23656  1020 ?        Ss   11:12   0:00 cron
daemon    1249  0.0  0.0  19140   160 ?        Ss   11:12   0:00 atd
root      1264  0.0  0.0      0     0 ?        S<   11:12   0:00 [iprt]
root      1314  3.2  0.2 1032288 22264 ?       Sl   11:12   0:47 /usr/sbin/libvirtd -d
root      1344  0.0  0.0  20016   952 tty1     Ss+  11:12   0:00 /sbin/getty -8 38400 tty1
whoopsie  1353  0.0  0.0 434928  5224 ?        Ssl  11:12   0:00 whoopsie
root      1407  0.0  0.0 351736  3776 ?        SLsl 11:12   0:00 lightdm
root      1423  2.3  0.6 431788 53720 tty7     Ssl+ 11:12   0:34 /usr/bin/X -core :0 -seat seat0 -auth /var/run/li
root      1426  0.0  0.0 287468  4248 ?        Sl   11:12   0:00 /usr/lib/accountsservice/accounts-daemon
root      1491  0.0  0.0  10232  3724 ?        S    11:12   0:00 /sbin/dhclient -d -sf /usr/lib/NetworkManager/nm-
nobody    1529  0.0  0.0  35224  1544 ?        S    11:12   0:00 /usr/sbin/dnsmasq --no-resolv --keep-in-foregroun
root      1575  0.0  0.0      0     0 ?        S    11:12   0:00 [kauditd]
root      1705  0.1  0.1  72056 10964 ?        Ssl  11:12   0:01 /opt/teamviewer9/tv_bin/teamviewerd -f
root      1790  0.0  0.0 170488  3876 ?        Sl   11:13   0:00 lightdm --session-child 12 19
root      1839  0.0  0.0 239448  4744 ?        Sl   11:13   0:00 /usr/lib/upower/upowerd
rtkit     1851  0.0  0.0 168916  1308 ?        SNl  11:13   0:00 /usr/lib/rtkit/rtkit-daemon
colord    2107  0.0  0.0 301564  7868 ?        Sl   11:13   0:00 /usr/lib/colord/colord
root      2246  0.0  0.0  77104  4108 ?        Ss   11:13   0:00 /usr/sbin/cupsd -f
libvirt+  2266  0.0  0.0  28208   964 ?        S    11:13   0:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dn
lp        2268  0.0  0.0  63156  1984 ?        S    11:13   0:00 /usr/lib/cups/notifier/dbus dbus://
eric      2283  0.0  0.0 370644  3836 ?        Sl   11:14   0:00 /usr/bin/gnome-keyring-daemon --daemonize --login
eric      2307  0.0  0.0  40204  2600 ?        Ss   11:14   0:00 init --user
eric      2311  0.0  0.0  10616   508 ?        Ss   11:14   0:00 ssh-agent
eric      2386  0.0  0.0  40024  2196 ?        Ss   11:14   0:00 dbus-daemon --fork --session --address=unix:abstr
eric      2395  0.0  0.0  22300  1156 ?        Ss   11:14   0:00 upstart-event-bridge
eric      2398  0.0  0.0  78200  4624 ?        Ss   11:14   0:00 /usr/lib/x86_64-linux-gnu/hud/window-stack-bridge
eric      2416  0.0  0.0  22308   396 ?        S    11:14   0:00 upstart-dbus-bridge --daemon --session --user --b
eric      2417  0.0  0.0  22308   396 ?        S    11:14   0:00 upstart-dbus-bridge --daemon --system --user --bu
eric      2421  0.0  0.0  30784   480 ?        S    11:14   0:00 upstart-file-bridge --daemon --user
eric      2428  0.1  0.0 365256  8076 ?        Ssl  11:14   0:01 /usr/bin/ibus-daemon --daemonize --xim
eric      2443  0.0  0.2 813556 18232 ?        Ssl  11:14   0:00 /usr/lib/unity-settings-daemon/unity-settings-dae
eric      2446  0.0  0.3 712324 25324 ?        Ssl  11:14   0:00 /usr/lib/x86_64-linux-gnu/hud/hud-service
eric      2449  0.0  0.0 337592  5420 ?        Ssl  11:14   0:00 /usr/lib/at-spi2-core/at-spi-bus-launcher --launc
eric      2450  0.0  0.1 649196 13316 ?        Ssl  11:14   0:00 gnome-session --session=ubuntu
eric      2455  0.1  0.2 575952 18840 ?        Ssl  11:14   0:01 /usr/lib/unity/unity-panel-service
eric      2457  0.0  0.0  39248  1976 ?        S    11:14   0:00 /bin/dbus-daemon --config-file=/etc/at-spi2/acces
eric      2465  0.0  0.0 124912  3316 ?        Sl   11:14   0:00 /usr/lib/at-spi2-core/at-spi2-registryd --use-gno
eric      2472  0.0  0.0 196640  3148 ?        Sl   11:14   0:00 /usr/lib/gvfs/gvfsd
eric      2478  0.0  0.0 345664  3120 ?        Sl   11:14   0:00 /usr/lib/gvfs/gvfsd-fuse /run/user/1000/gvfs -f -
eric      2487  0.0  0.0 280880  5500 ?        Sl   11:14   0:00 /usr/lib/ibus/ibus-dconf
eric      2499  0.0  0.1 483580 14980 ?        Sl   11:14   0:00 /usr/lib/ibus/ibus-ui-gtk3
eric      2503  0.0  0.0 386820  7356 ?        Sl   11:14   0:00 /usr/lib/ibus/ibus-x11 --kill-daemon
eric      2506  0.0  0.2 557380 17316 ?        Sl   11:14   0:01 /usr/lib/x86_64-linux-gnu/bamf/bamfdaemon
eric      2516  0.0  0.0 205148  5444 ?        Sl   11:14   0:00 /usr/lib/ibus/ibus-engine-simple
eric      2554  0.0  0.0 335476  3428 ?        Ssl  11:14   0:00 /usr/lib/x86_64-linux-gnu/indicator-messages/indi
eric      2555  0.0  0.1 591136 13216 ?        Sl   11:14   0:00 /usr/lib/x86_64-linux-gnu/indicator-keyboard-serv
eric      2556  0.0  0.0 263640  2904 ?        Ssl  11:14   0:00 /usr/lib/x86_64-linux-gnu/indicator-bluetooth/ind
eric      2558  0.0  0.0 278584  5344 ?        Ssl  11:14   0:00 /usr/lib/x86_64-linux-gnu/indicator-power/indicat
eric      2561  0.0  0.1 770068 14012 ?        Ssl  11:14   0:00 /usr/lib/x86_64-linux-gnu/indicator-datetime/indi
eric      2567  0.0  0.0 479388  5948 ?        Ssl  11:14   0:00 /usr/lib/x86_64-linux-gnu/indicator-sound/indicat
eric      2568  0.0  0.1 446976 13460 ?        Ssl  11:14   0:00 /usr/lib/x86_64-linux-gnu/indicator-printers/indi
eric      2580  0.0  0.0 521428  6384 ?        S<l  11:14   0:00 /usr/bin/pulseaudio --start --log-target=syslog
eric      2581  0.0  0.0 762660  4000 ?        Ssl  11:14   0:00 /usr/lib/x86_64-linux-gnu/indicator-session/indic
eric      2593  0.0  0.0 360480  7084 ?        Ssl  11:14   0:00 /usr/lib/x86_64-linux-gnu/indicator-application/i
eric      2615  0.0  0.0 178308  2768 ?        Sl   11:14   0:00 /usr/lib/dconf/dconf-service
eric      2626  0.0  0.1 480968 10688 ?        Sl   11:14   0:00 /usr/lib/evolution/evolution-source-registry
eric      2628  0.0  0.0  20232   940 ?        S    11:14   0:00 syndaemon -i 1.0 -t -K -R
eric      2658  0.0  0.1 350296 15524 ?        Sl   11:14   0:00 /usr/lib/x86_64-linux-gnu/notify-osd
eric      2668  1.7  0.8 1283036 66452 ?       Sl   11:14   0:24 compiz
eric      2724  0.0  0.5 803404 46068 ?        Sl   11:14   0:00 /usr/lib/evolution/evolution-calendar-factory
eric      2730  0.0  0.1 403376  9512 ?        Sl   11:14   0:00 /usr/lib/unity-settings-daemon/unity-fallback-mou
eric      2731  0.0  0.2 674704 20268 ?        Sl   11:14   0:00 nm-applet
eric      2740  0.0  0.1 331704  9504 ?        Sl   11:14   0:00 /usr/lib/policykit-1-gnome/polkit-gnome-authentic
eric      2743  0.0  0.3 830076 31016 ?        Sl   11:14   0:00 nautilus -n
eric      2747  0.0  0.0 372704  5680 ?        Sl   11:14   0:00 /usr/lib/gvfs/gvfs-udisks2-volume-monitor
eric      2755  0.0  0.0  58260  3720 ?        S    11:14   0:00 /usr/lib/x86_64-linux-gnu/gconf/gconfd-2
root      2764  0.0  0.0 437148  7864 ?        Sl   11:14   0:00 /usr/lib/udisks2/udisksd --no-debug
eric      2773  0.0  0.0 212568  3436 ?        Sl   11:14   0:00 /usr/lib/gvfs/gvfs-gphoto2-volume-monitor
eric      2780  0.0  0.0 285964  3352 ?        Sl   11:14   0:00 /usr/lib/gvfs/gvfs-afc-volume-monitor
eric      2785  0.0  0.0 200276  2848 ?        Sl   11:14   0:00 /usr/lib/gvfs/gvfs-mtp-volume-monitor
eric      2799  0.0  0.0 367604  5888 ?        Sl   11:14   0:00 /usr/lib/gvfs/gvfsd-trash --spawner :1.7 /org/gtk
eric      2805  0.0  0.0 270376  5100 ?        Sl   11:14   0:00 /usr/lib/gvfs/gvfsd-burn --spawner :1.7 /org/gtk/
eric      2837  4.0  1.2 1481612 102828 ?      Ssl  11:14   0:54 python /usr/share/virt-manager/virt-manager.py
eric      2842  0.0  0.1 455016 11996 ?        Sl   11:14   0:00 telepathy-indicator
eric      2849  0.0  0.0 322476  7344 ?        Sl   11:14   0:00 /usr/lib/telepathy/mission-control-5
eric      2867  0.0  0.1 467336  9200 ?        Sl   11:14   0:00 zeitgeist-datahub
eric      2872  0.0  0.0 348872  4580 ?        Sl   11:14   0:00 /usr/bin/zeitgeist-daemon
eric      2878  0.0  0.1 252036  9412 ?        Sl   11:14   0:00 /usr/lib/x86_64-linux-gnu/zeitgeist-fts
eric      2890  0.0  0.0  11412   616 ?        S    11:14   0:00 /bin/cat
root      2990  0.0  0.0      0     0 ?        S<   11:15   0:00 [kworker/u5:2]
eric      2997  0.0  0.1 498104 13224 ?        Sl   11:15   0:00 update-notifier
eric      3047  0.2  0.2 653160 20868 ?        Sl   11:15   0:02 gnome-terminal
root      3054  0.0  0.0      0     0 ?        S<   11:15   0:00 [dio/sdb3]
eric      3058  0.0  0.0  14824   812 ?        S    11:15   0:00 gnome-pty-helper
eric      3059  0.0  0.0  26952  3924 pts/0    Ss   11:15   0:00 bash
eric      3147  0.0  0.0 377868  4168 ?        Sl   11:16   0:00 /usr/lib/x86_64-linux-gnu/deja-dup/deja-dup-monit
root      3911  0.0  0.0      0     0 ?        S    11:25   0:00 [kworker/u4:0]
root      3966  0.0  0.0      0     0 ?        S    11:26   0:00 [kworker/u4:1]
root      4074  0.0  0.0      0     0 ?        S    11:28   0:00 [kworker/0:2]
root      4283  0.0  0.0      0     0 ?        S    11:30   0:00 [kworker/1:1]
libvirt+  4384 53.1 26.2 2642208 2131732 ?     Sl   11:31   3:10 qemu-system-x86_64 -enable-kvm -name win7dns -S -
root      4386  0.0  0.0      0     0 ?        S    11:31   0:00 [kvm-pit/4384]
root      4690  0.6  0.0      0     0 ?        S    11:35   0:00 [kworker/1:2]
root      4694  1.5  0.0      0     0 ?        S    11:36   0:01 [kworker/0:1]
eric      4720 16.5  2.9 1022024 240172 ?      Sl   11:36   0:09 /usr/lib/thunderbird/thunderbird
root      4795  0.0  0.0      0     0 ?        S    11:36   0:00 [kworker/u4:2]
eric      4848  0.0  0.0  22644  1292 pts/0    R+   11:37   0:00 ps -aux
eric@garnet:~$

I don't develop the management tool, but there are several. The most advanced UI is probably in GNOME Boxes, but it also has less functionality than virt-manager.

Paolo

next puzzle. updates are not working
using bridged to eth0
using virt io driver (checked install on windows)
browser works in vm (quite well in fact)
watching output of tcpdump

and there is no apparent traffic for updates.

any ideas?

a little more info

in resource manager, svchost.exe (netsvcs) is running at 100%

The fine magic comes in three forms. Keystroke injection, context feedback, and exporting UI elements such as microphone level, recognition correction, partial recognition pop-ups into the linux environment.

All of these have in common the magic trick of using the isolation of the Windows environment to provide a single dictation target to NaturallySpeaking. All of the information necessary for the above capabilities would pass through this target. initially, this would be an ssh session with command redirecting standard into whatever accessibility inputs available.

The host side of this gateway would be responsible for all of the
proper input redirection. In theory, it would even be possible to direct speech recognition towards two targets depending on the grammar. For example in the programming by speech environment I'm working on, I would dictate directly into the editor sometimes and into a secondary window for focused speech UI action. At no time, would my hand touch the mouse. :-) It will happen because of the context set by the speech UI as a deliberate effect of certain commands.

--- longer ramble about speech and nuance issues. ---

Being a crip who's trying to write code with speech, it's not going to be fast. once I get the Basic keystroke injection working, it will be good enough to continuing developing my program by speech environment. But to discuss that, would go down the rathole of current models of speech user interfaces, why don't work, things you shouldn't do such as speaking the keyboard, intentional automation, contextual grammars and a host of other things of spent the past 15 years learning about and figuring out how to make a change. By the way, that knowledge and passion is why I I've started a consulting practice that focuses on improving user experience/user interfaces starting from the intent of the user and perspective of a disabled person with the result being an improved UI for everybody.

The hardest part is going to be everything except a keystroke injection. This is because they require special knowledge that nuance is loath to give up. I don't get it. Nuance totes and gets federal benefits for producing something that is "section 508 compliant" yet, the only way you could be considered an accessibility tool is if you do nothing but write in Microsoft Word. I worked for a dragon reseller for a while with medical record systems and, nuance doesn't even make an attempt to try and speech enable the medical record environment. They have people using a couple of solutions that don't work well and effectively provide no UI automation[1] tied into speech commands.

A bunch of us techno Crips have built environments that greatly enhance the range of solutions NaturallySpeaking could be used for but, nuance won't talk to us, won't give us any documentation to keep things running on our own, won't sell us the documentation either and worst of all, they have written terms into the AUP designed to bar extensions like our environment unless you buy the most expensive version of NaturallySpeaking available.

And did I mention that they have many bugs that are a significant problem for every user, not to mention the scripts and the last time I checked, it will cost about $10 to report a bug (support call cost) and then there's no guarantee they'll ever fix. In version 13, I'm seeing bugs that have been around since version 7 or 8.

I will do what I can to implement the magic and when I get stumped, then, I'll figure out what I'm going to do technically and politically.

--- eric

[1] This is kind of a lie. They have the tools to what you navigate blindly through an application (i.e. hit 15 tabs, two down arrows, and a mouse click and it might end up in the right UI element to do something. unfortunately, they do not have anything to make it predictable, repeatable or survive revisions in the user interface. But this is one of those rat holes I said it wouldn't go down.