input/speaker: additional clicks and tones for future accessibility

From: Karl Dahlke <[email protected]>

Add clicks, tones, and tone sequences to the pc speaker.

The speaker driver can play a tone at a specified frequency,
or the standard control G bell,
which is a special case of TONE at 1000 hz 0.1 seconds.
Add kd_mkpulse() to generate a soft click.
This is introduced to support accessibility modules and adapters in the future. It is a means to an end.
With this function in place, a module can easily provide soft clicks, i.e. audible feedback, whenever a key is depressed,
or when that keystroke is echoed on screen.
This allows a blind user, for example, to have rapid feedback while typing, even if he is, at the same time, listening to text that is already on screen. It is faster and more convenient than having characters echoed verbally. And it works all the time, even if speech or braille is not working for whatever reason.

Also add the function kd_mknotes,
which plays a series of tones in the background.
Of course this can already be done with kd_mksound and timers, but why should everyone reinvent the wheel?
It is better to write the function once, properly, in the kernel, and let modules use it thereafter.
Again, this is a means to an end.
Accessibility modules can generate rapid sequences of notes
to indicate various conditions, sometimes error conditions, especially if speech or braille is not working.
These notes may be the only feedback the user has to diagnose the problem. This may be useful to other developers in other situations as well.

Finally add kd_mksteps to run something like a chromatic scale, from one frequency to another in specified steps.
The half-tone scale, with a step of 6%, starting at middle C, is approximately kd_mksteps(260, 530, 6, 150);

---
Built and tested on 3.17.4, the latest stable kernel.

@@ -18,6 +18,8 @@
 #include <linux/input.h>
 #include <linux/platform_device.h>
 #include <linux/timex.h>
+#include <linux/delay.h>
+
 #include <asm/io.h>
 
 MODULE_AUTHOR("Vojtech Pavlik <[email protected]>");

@@ -25,25 +27,26 @@ MODULE_DESCRIPTION("PC Speaker beeper dr
 MODULE_LICENSE("GPL");
 MODULE_ALIAS("platform:pcspkr");
 
-static int pcspkr_event(struct input_dev *dev, unsigned int type, unsigned int code, int value)
+/* Toggle the speaker, but not if a tone is sounding */
+static void speaker_toggle(void)
 {
-	unsigned int count = 0;
+	char c;
 	unsigned long flags;
 
-	if (type != EV_SND)
-		return -1;
-
-	switch (code) {
-		case SND_BELL: if (value) value = 1000;
-		case SND_TONE: break;
-		default: return -1;
+	raw_spin_lock_irqsave(&i8253_lock, flags);
+	c = inb_p(0x61);
+	if ((c&3) != 3) {
+		c &= 0xfe;
+		c ^= 2; /* toggle */
+		outb(c, 0x61);
 	}
+	raw_spin_unlock_irqrestore(&i8253_lock, flags);
+}
 
-	if (value > 20 && value < 32767)
-		count = PIT_TICK_RATE / value;
-
+static void speaker_sing(unsigned int count)
+{
+	unsigned long flags;
 	raw_spin_lock_irqsave(&i8253_lock, flags);
-
 	if (count) {
 		/* set command for counter 2, 2 byte write */
 		outb_p(0xB6, 0x43);

@@ -56,8 +59,44 @@ static int pcspkr_event(struct input_dev
 		/* disable counter 2 */
 		outb(inb_p(0x61) & 0xFC, 0x61);
 	}
-
 	raw_spin_unlock_irqrestore(&i8253_lock, flags);
+}
+
+static int pcspkr_event(struct input_dev *dev, unsigned int type, unsigned int code, int value)
+{
+	unsigned int count;
+
+	if (type != EV_SND)
+		return -1;
+
+	switch (code) {
+	case SND_BELL:
+		if (value)
+			value = 1000;
+		/* fall through */
+
+	case SND_TONE:
+		count = 0;
+		if (value > 20 && value < 32767)
+			count = PIT_TICK_RATE / value;
+		speaker_sing(count);
+		break;
+
+	case SND_PULSE:
+		/* Don't hold the cpu for more than 1ms */
+		/* no reason for a pulse to ever be longer than that. */
+		if (value > 1000)
+			value = 1000;
+		/* if a pulse is too short it can't be heard anyways. */
+		if (value < 20)
+			break;
+		speaker_toggle();
+		udelay(value);
+		speaker_toggle();
+		break;
+
+	default: return -1;
+	}
 
 	return 0;
 }

@@ -80,7 +119,8 @@ static int pcspkr_probe(struct platform_
 	pcspkr_dev->dev.parent = &dev->dev;
 
 	pcspkr_dev->evbit[0] = BIT_MASK(EV_SND);
-	pcspkr_dev->sndbit[0] = BIT_MASK(SND_BELL) | BIT_MASK(SND_TONE);
+	pcspkr_dev->sndbit[0] = BIT_MASK(SND_BELL) | BIT_MASK(SND_TONE) |
+		BIT_MASK(SND_PULSE);
 	pcspkr_dev->event = pcspkr_event;
 
 	err = input_register_device(pcspkr_dev);

@@ -28,6 +28,9 @@
 #endif
 
 extern void kd_mksound(unsigned int hz, unsigned int ticks);
+extern void kd_mkpulse(unsigned int width);
+extern void kd_mknotes(const short *p);
+extern void kd_mksteps(int freq1, int freq2, int step, int duration);
 extern int kbd_rate(struct kbd_repeat *rep);
 extern int fg_console, last_console, want_console;
 

@@ -934,6 +934,7 @@ struct input_keymap_entry {
 #define SND_CLICK		0x00
 #define SND_BELL		0x01
 #define SND_TONE		0x02
+#define SND_PULSE		0x03
 #define SND_MAX			0x07
 #define SND_CNT			(SND_MAX+1)
 

Hi Karl,

I have exactly the same response as a year earlier:

"I do not think it is a good idea to add SND_PULSE as it can be easily implemented by SND_TONE with the additional benefit that parameters of the click can be adjusted. Also, if clicking is done elsewhere, it would work with other speaker drivers besides pcspkr.

...

[re kd_mknotes] Can it be put into a library instead? Especially given David's work on trying to push the VT code out of the kernel. Also, what if you want clicks to go through sound card and not the speaker interface?"

In other words, I'd rather have it all be done in userpsace by a daemon that listens to all keystrokes and emits notes or clicks or something else via whatever sound device is present on the system, be it a PC speaker, SPARC speaker, a sound card, or maybe something else.

Thanks.

Dmitry

Thank you for your thoughtful reply.

It isn't that simple.
Example: if a tone is playing for any reason the clicks do not interrupt the tone. (It would sound terrible, I've tried it).
Clicks and tones are different in spirit and in implementation, almost orthogonal, and my code manages all this.

But you see, in my world, I precisely don't want that to happen, because sometimes the sound card isn't working.
There are lots of reasons a sound card might not work,
alsa and modules and pulseaudio are more finicky than you might think, not to mention the speech adapters that sit on top of them.
If any link in this chain is broken, I get no speech, period.
If I then don't get the clicks or tones either, I'm really lost at sea. Thus the modules that I write,
and that other people write as well,
specifically send small diagnostics through the pc speaker, because that just might be all the feedback we have.
Here is a good example, grub, or maybe it's grub2,
can't reasonably be expected to work through a sound card,
so it can actually generate morse code at the pc speaker to help us load the desired OS.
It's just a different philosophy that we have I guess.
So instead of reinventing these routines over and over I thought it might be useful to have some of this common functionality in the pcspkr driver in the kernel.
I mean it's already halfway there, this is just the other half. All managed just like tones are managed today.
It's just the icing on the cake.

Respectfully,

Karl Dahlke

Even if it sounds horrible why would not you want feedback for key press if there is tone? Would you not rather want to know that key press has been registered?

I understand that you might not want to use sound card, that does not mean that nobody else would now want it. There are boxes without pcspkr or where speaker is provided by different module.

You should be able to work directly with ALSA though, right? That would cut some of the stack away.

But grub is not an OS, so naturally it does not have sound card support. It also does not support any of the haptic devices that might be connected to the box but that does not mean that haptic devices are not suitable as feedback devices.

I am sorry, but I still believe that doing this in pcspkr/tty would be doing this at the wrong level and userspace solution would be the right one. It does not need to use real sound card if you prefer not to, but I do not think it should prohibit using it or any other device that can provide adequate feedback.

Thanks.

Dmitry

SND_PULSE would make sense in case (and only in case) there are devices that can emit that kind of sound without being able to implement SND_TONE.

This is the reason why SND_CLICK and SND_BELL exist.

And, by the way, the implementation of SND_PULSE in the patch is quite horrible. If a variable-width pulse makes sense at all (based on the speaker dynamics, a long enough pause will be two clicks and high power consumption inbetween), it should be done by programming the i8253 to do a one-shot timer rather than bitbanging it.

Vojtech Pavlik
Director SUSE Labs

It seems to be the general consensus that these sound functions should not be integrated into the kernel - and that's ok.
Allow me then to ask one more question if I may.
I have a module, ttyclicks.ko, that makes sounds and clicks and such, and is used primarily by adapters.
By design, it operates the pc speaker, and is never intended
to vector through a sound card or other such.
In fact it provides feedback when the sound card is not working. That might eliminate some of your concerns.
My question is, would it be possible to add this module to the kernel tree? If you wanted to add it under staging, that would be fine. After all, the Speakup modules are all under staging,
and since so few people use them, that's not a problem.
My modules would also be used by a slim minority,
but that minority would sure find it convenient to have the modules distributed with the kernel.
If you think this is at least a possibility, I will put a patch together. If not, I will maintain them privately, as I already do.
To clarify how the module works,
here is the file that would be added as Documentation/accessibility/ttyclicks.txt

Thank you for your consideration.

======================================================================

The tty clicks module

			---------------------
			Karl Dahlke
			[email protected]

When this module is installed, soft clicks accompany
the nonspace characters that are sent to the console,
while a longer swoop indicates a newline character.
These sounds simulate an old fashion teletype running at 2400 baud. Why would you want such a thing?
You probably wouldn't, but a blind person might.
He can tell, by clicks alone, when the computer responds to a command, and he can discern the quantity and format of that response, without any speech or braille.
In addition, the output is throttled, and does not fly by the screen in a flash, faster than one could possibly hit control S.

This module is separate from any sound cards, thus providing valuable audio feedback if the sound card is not working.
It can be loaded very early in the boot sequence,
since it does not depend on anything else.

This module uses the in-built toggle speaker at port 0x61. Not all machines have this speaker, but most do. If the speaker is lacking, the module is a no-op.

This module will not click and chirp if you are in X.
It responds to text characters that pass through the standard tty, and will not do anything in graphics mode.
Even the terminal emulator, running under a desktop, still runs through X, and won't work with ttyclicks.
Switch to a text console to hear the clicks.

Module parameters:

As always, you can run modinfo ttyclicks.ko to review these parameters.

enabled = 1 or 0

If 1, then sounds are enabled as soon as the module is loaded. If 0, then sounds are not enabled.
These sounds can be turned on or off at run time by the exported symbol

	bool ttyclicks_on;

Thus other modules can turn sounds on or off.
In particular, any adapter that includes a lodable module has access to this master switch.

fgtty = 1 or 0

This parameter enables the clicks that accompany the display of characters from the foreground tty.
The corresponding exported symbol is

	bool ttyclicks_tty;

kmsg = 1 or 0

Printk messages do not pass through the tty,
and do not generate the aforementioned clicks.
However, if this switch is enabled,
each printk message generates a distinctive sequence of tones. This is like an alarm,
telling the user that he needs to read the message on the screen. The corresponding exported symbol is

	bool ttyclicks_kmsg;

cursormotion = 0 or 1

Many screen programs generate ansi escape sequences that position the cursor and set character attributes such as underline etc.
This is not true output, and does not generate clicks by default. However, you can set this parameter to 1 if you want these characters to generate clicks.

Exported functions.
These are all subject to the master switch ttyclicks_on.

void ttyclicks_click(void);

Generate a 0.6 millisecond pulse.

void ttyclicks_cr(void);

Generate a quick frequency swoop from 2900 to 3600 hz.
This sound is associated with the newline character at the console, but you can use this function to generate the same sound whenever you wish.

void ttyclicks_notes(const short *notelist);

Play a series of notes in the background.
Each note is defined by two shorts, a frequency and a duration. The frequency is in hurtz, and the duration is in hundredths of a second. A frequency of -1 is a rest.
A frequency of 0 terminates the array.
The queue can only hold 64 notes, so don't try to play an entire sonata.

void ttyclicks_steps(int freq1, int freq2, int step, int duration)

Play a series of notes, of the specified duration, moving from frequency freq1 to frequency freq2 in a geometric progression of step%. This is a customizable chromatic scale.
The traditional scale, based on half tones, and starting at middle C, has an approximate step of 6%, and looks like this. ttyclicks_mksteps(260, 530, 6, 150);
Use a negative step for a descending scale.

void ttyclicks_bell(void);

This function generates an 1800 hz tone for a tenth of a second. this is associated with the control G bell.
The traditional bell is 1000 hz, but I wanted a slightly higher tone. As long as this module is loaded, it swallows control G, and sounds the bell, rather then passing it through to the console and letting the console ring the bell. This allows the master switch, ttyclicks_on,
to enable or disable the control G bell along with all the other sounds. You can turn off all the sounds in one go, if they are bothering your roommate.

This module is integral to many command line adapters,
as a complement to speech or braille. Beyond this,
it can prove invaluable if the adapter is not working for any reason.