SCO. Some ideas.

As SCO over bluetooth is only really suitable for audio sound data, I see no real problem with alsa being the only interface in linux to send data over a bluetooth SCO connection. (only the actually sco data, not the bluetooth profile control)
Does anyone see a problem with that?

If people think we should also make it easy to add an OSS driver, and also a network socket driver, we could maybe work out some devision between the SCO specifics and the ALSA specifics. But for a first attempt, we should keep the SCO and ALSA driver tightly matched.

Therefore maybe making the bluetooth sco and alsa-sco module being the same module, with an ioctl interface so that a user space application can handle the creation of bluetooth profile SCO pairing.

For example, if using a Headset profile, the alsa PCM and alsa MIXER device would only appear if the userspace application had already set up the RFCOMM connections. The userspace application would also have to have set up the bluetooth specific details for the SCO connection, so that when alsa opens the PCM, it has enough information to open a SCO connection.

I have looked at this for the alsa->sco->bluetooth->hci_usb drivers. The alsa driver has to be able to do the following: -
1) open - this should check that all bluetooth connections are up that need to be up in order for SCO data to pass. It could do this via interaction with the userspace bluetooth profile.

2) close - close everything neatly.

3) hw_params - allocate buffers etc. for the pcm audio as well as the hw_params config. Can be called multiple times, so re-alloc of buffers should be allowed for. hw_params are things like sample rate, number of channels, PCM format. Obviously there are limits on what these values can take when using bluetooth. 16bit PCM, 8bit PCM, 8bit A-law, 8bit u-Law. Start by also limiting it to 1 PCM channel. I.E. Mono and not stereo.

4) pcm_prepare - actually set the hw_params. e.g. do the equivalent of hciconfig hci0 voice XX, where XX depends on the hw_params. Also selecting usb alt profiles.
The only difference between hw_params and pcm_prepare is that pcm_prepare is called for xrun recovery.

5) trigger - actually start/stop usb_urbs (e.g. call usb_submit_urb/usb_unlink_urb now.)

6) pointer - get hw_usb_frame_pointer and modify the result to simulate an audio ring buffer of the size configured in the hw_params. When retrieving the pointer, one should also retrieve the valid range that the pointer can have, so one can adjust for pointer wrap around. For usb, this would be the usb_get_current_frame_number() for the pointer, and dev->bus->iso_sched_frames for the range of values it can take.

7) period_time_elapsed - hci_usb should call this on each urb_complete call. For our use, it seems sensible to make 1 urb == 1 alsa period.

If we can let the alsa bluetooth audio driver have access to the hardware pointers (in this case the usb current frame pointers), we will get a much more accurate idea of exactly which audio sample is currently being played.

Also, letting alsa control period and buffer sizes, and getting that to directly determine usb urb sizes, we would give complete control of buffer sizes to the application, which is vital for low latency applications like Voice over IP.

Summary: -
we need to provide up to the sco level, control of low level usb interface isoc parameters. E.g. prepare, trigger, pointer, period_time_elapsed, hw_params.

If people are happy with this proposal, I will start writing the alsa driver, and also start adding to the bluetooth -> hci_usb driver api to allow for it. The api will only change for SCO data handling, as Bulk and Int handling don't need such fine controls.

Hi James,

Quoted 2 lines

As SCO over bluetooth is only really suitable for audio sound data, I see no real problem with alsa being the only interface in linux to send data over a bluetooth SCO connection. (only the actually sco data, not the bluetooth profile control) Does anyone see a problem with that?

there are two interfaces with Bluetooth 1.2. SCO/eSCO audio and eSCO data. This means we need an ALSA interface and a socket interface.

Quoted 3 lines

If people think we should also make it easy to add an OSS driver, and also a network socket driver, we could maybe work out some devision between the SCO specifics and the ALSA specifics. But for a first attempt, we should keep the SCO and ALSA driver tightly matched.

We need the socket driver, but we don't need an OSS driver. But I see no problem with this, because the RFCOMM layer also includes socket and TTY support at the same time.

Quoted 1 line

Therefore maybe making the bluetooth sco and alsa-sco module being the same module, with an ioctl interface so that a user space application can handle the creation of bluetooth profile SCO pairing.

Yes. Only one kernel module called sco.

Quoted 3 lines

For example, if using a Headset profile, the alsa PCM and alsa MIXER device would only appear if the userspace application had already set up the RFCOMM connections. The userspace application would also have to have set up the bluetooth specific details for the SCO connection, so that when alsa opens the PCM, it has enough information to open a SCO connection.

ALSA don't open the SCO connection. This is part of the application that implements the profile. In the case of the headset profile, you must first have the RFCOMM connection and then the SCO link. No SCO link without an ACL link.

Quoted 5 lines

I have looked at this for the alsa->sco->bluetooth->hci_usb drivers.
The alsa driver has to be able to do the following: -
1) open - this should check that all bluetooth connections are up that need to be up in order for SCO data to pass. It could do this via interaction with the userspace bluetooth profile.

If there is no SCO link -> failure.

Quoted 3 lines

2) close - close everything neatly.

Of course, but the SCO link should be closed by the application.

Quoted 3 lines

3) hw_params - allocate buffers etc. for the pcm audio as well as the hw_params config. Can be called multiple times, so re-alloc of buffers should be allowed for. hw_params are things like sample rate, number of channels, PCM format. Obviously there are limits on what these values can take when using bluetooth. 16bit PCM, 8bit PCM, 8bit A-law, 8bit u-Law. Start by also limiting it to 1 PCM channel. I.E. Mono and not stereo.

The possible values depends on the voice setting.

Quoted 5 lines

4) pcm_prepare - actually set the hw_params. e.g. do the equivalent of hciconfig hci0 voice XX, where XX depends on the hw_params. Also selecting usb alt profiles.
The only difference between hw_params and pcm_prepare is that pcm_prepare is called for xrun recovery.

I am not sure if we should allow the ALSA driver to change the voice settings. As I know it is not possible to change the voice setting of an already existing SCO link.

Quoted 1 line

5) trigger - actually start/stop usb_urbs (e.g. call usb_submit_urb/usb_unlink_urb now.)

Not part of the ALSA driver. This is a problem of the HCI USB driver.

Quoted 5 lines

6) pointer - get hw_usb_frame_pointer and modify the result to simulate an audio ring buffer of the size configured in the hw_params. When retrieving the pointer, one should also retrieve the valid range that the pointer can have, so one can adjust for pointer wrap around. For usb, this would be the usb_get_current_frame_number() for the pointer, and dev->bus->iso_sched_frames for the range of values it can take.

The SCO interface is not driver specific. So no interaction we the HCI USB driver.

Quoted 7 lines

7) period_time_elapsed - hci_usb should call this on each urb_complete call. For our use, it seems sensible to make 1 urb == 1 alsa period.

If we can let the alsa bluetooth audio driver have access to the hardware pointers (in this case the usb current frame pointers), we will get a much more accurate idea of exactly which audio sample is currently being played.

Also, letting alsa control period and buffer sizes, and getting that to directly determine usb urb sizes, we would give complete control of buffer sizes to the application, which is vital for low latency applications like Voice over IP.

No. ALSA <-> SCO is driver and hardware independent.

Quoted 6 lines

Summary: -
we need to provide up to the sco level, control of low level usb interface isoc parameters. E.g. prepare, trigger, pointer, period_time_elapsed, hw_params.

If people are happy with this proposal, I will start writing the alsa driver, and also start adding to the bluetooth -> hci_usb driver api to allow for it. The api will only change for SCO data handling, as Bulk and Int handling don't need such fine controls.

Let me repeat it. The upper interface to SCO is driver and hardware independent.

We need to extend the HCI USB driver to support start/stop of ISOC URB's on demand and to select the right alternate settings. And this means to split the hash for number of ACL/SCO connections and send a notification to the driver if a new connection is created/released of if the voice settings are changed. We already talked about this stuff. I had already a patch for it, but the hash split was missing.

The ALSA <-> SCO driver should be written with the current API provided by the HCI layer. Check how the current SCO driver gets and sends its data packets. Start with this driver and don't worry so much about the USB specific part. Use voice setting 0x0060 as the only supported value at the moment.

Regards

Marcel

This is something I need to be able to understand better. What should actually be adjusting the voice setting? To me it seems sensible that we should try to get the voice setting to match the alsa PCM samples if we can.
Of course, another approach is getting the application to set the voice setting, and then open the SCO link, and then force alsa to use whatever voice setting the SCO link was set to. We would get the application to inform alsa about this voice setting via an ioctl.

The alsa-driver receives a trigger() api call from an alsa based application. It would be nice to pass this trigger() function on down the stack to whichever bluetooth hci driver we are using (hci_usb or whatever else), and that hci driver handle it appropriately. I just gave the usb_submit_urb() as an example of what the hci_usb.c driver would do when it received the trigger() call.
For example. The SCO link is open, but no actual sound samples start being sent or received from the CPU until the trigger() call.

I was using the HCI USB driver as an example only.
The alsa-driver receives a pointer() from the alsa based application. we need to pass a call to the HCI driver (hardware independently) which would return the frame pointer. I don't think the concept of a frame_pointer is USB specific. Any Bluetooth hardware driver will use a frame_pointer.

Maybe I should look at the patch first, but ..

I cannot see anywhere in the current HCI layer api where I can find out which HCI SCO packet is currently(in realtime) being output. All I know is that if I send an HCI SCO packet down the stack, it will get queued, and at some point be output. That is not good enough.

Hi James,

Quoted 3 lines

This is something I need to be able to understand better.
What should actually be adjusting the voice setting? To me it seems sensible that we should try to get the voice setting to match the alsa PCM samples if we can.
Of course, another approach is getting the application to set the voice setting, and then open the SCO link, and then force alsa to use whatever voice setting the SCO link was set to. We would get the application to inform alsa about this voice setting via an ioctl.

we only need to know about the input coding. The ALSA driver maybe needs to convert the PCM samples to the input coding of the Bluetooth device.

Quoted 5 lines

The alsa-driver receives a trigger() api call from an alsa based application. It would be nice to pass this trigger() function on down the stack to whichever bluetooth hci driver we are using (hci_usb or whatever else), and that hci driver handle it appropriately. I just gave the usb_submit_urb() as an example of what the hci_usb.c driver would do when it received the trigger() call.
For example. The SCO link is open, but no actual sound samples start being sent or received from the CPU until the trigger() call.

I don't see any need for it.

Quoted 4 lines

I was using the HCI USB driver as an example only.
The alsa-driver receives a pointer() from the alsa based application. we need to pass a call to the HCI driver (hardware independently) which would return the frame pointer. I don't think the concept of a frame_pointer is USB specific. Any Bluetooth hardware driver will use a frame_pointer.

I don't get this either, because we don't need to interface with any hardware drivers.

Quoted 3 lines

Maybe I should look at the patch first, but ..

You can find it in the mailing list archive.

Quoted 1 line

I cannot see anywhere in the current HCI layer api where I can find out which HCI SCO packet is currently(in realtime) being output. All I know is that if I send an HCI SCO packet down the stack, it will get queued, and at some point be output. That is not good enough.

Why would you wanna know this? If you send too much data, the Bluetooth chip itself will drop the packets.

Regards

Marcel

alsa handles format conversion automatically, all we have to tell alsa is which formats we can do, and alsa does the rest. So, if alsa is not allowed to modify the hciconfig hci0 voice XX setting, it can just be told what it is via an IOCTL and then tell alsa to do conversions for any other formats.

Some audio applications need this "pointer" feature, which is why alsa provides it.
An example of this is playback of DVDs.
Video output needs to be kept closely in sync with the audio, and this "pointer" and "trigger" feature allow the application to closely control exactly when the sound plays. "trigger" handles the "start the sound exactly now", and "pointer" provides feedback, saying that "sound sample xyz is currently playing, so display video frame xyz now". "pointer" also provides for "audio is playing 0.25 seconds ahead of video, adjust it to match".
These are not hardware specific features, all sound hardware has these features, including bluetooth audio, and we just need some way for higher layers to get access to the values.
Another example is Voice over IP. If using a headset, one might want to

use it to make VoIP calls over the internet. One of the most important issues with VoIP is latency. The lower the latency, the more natural the conversation will be. Long latency produces conversations will long breaks in between and possible delayed echos. It is a lot more difficult to have a natually smooth conversation if the latency is high. These "pointer" and "trigger" and "period_time_elapsed" functions allow for applications to program minimum latancy that any particular hardware can handle.

Hi James,

Quoted 5 lines

Some audio applications need this "pointer" feature, which is why alsa provides it.
An example of this is playback of DVDs.
Video output needs to be kept closely in sync with the audio, and this "pointer" and "trigger" feature allow the application to closely control exactly when the sound plays. "trigger" handles the "start the sound exactly now", and "pointer" provides feedback, saying that "sound sample xyz is currently playing, so display video frame xyz now". "pointer" also provides for "audio is playing 0.25 seconds ahead of video, adjust it to match".
These are not hardware specific features, all sound hardware has these features, including bluetooth audio, and we just need some way for higher layers to get access to the values.
Another example is Voice over IP. If using a headset, one might want to use it to make VoIP calls over the internet. One of the most important issues with VoIP is latency. The lower the latency, the more natural the conversation will be. Long latency produces conversations will long breaks in between and possible delayed echos. It is a lot more difficult to have a natually smooth conversation if the latency is high. These "pointer" and "trigger" and "period_time_elapsed" functions allow for applications to program minimum latancy that any particular hardware can handle.

You shouldn't think so much about it. Try to implement the ALSA driver without it and we see how it goes.

Regards

Marcel

So you want me to implement it in such a way that would add 10-20ms of latency and have no audio/video syncronisation features?

We might as well stay with the current alsa over network sockets which is what we have now.

Hi James,

Quoted 1 line

So you want me to implement it in such a way that would add 10-20ms of latency and have no audio/video syncronisation features?

yes, do it.

Quoted 1 line

We might as well stay with the current alsa over network sockets which is what we have now.

We will see what the latency really is if we go directly from the ALSA driver to the HCI SCO interface.

Regards

Marcel

I have started to think of how we might better achieve our goal without explicitly having trigger/pointer etc. api calls from the HCI to the SCO layer.
The current bluez stack handles HCI SCO receiving ok for now. HCI SCO packets can be lost, but that is not so much of a problem. The current problem is the HCI SCO sending. i.e. CPU to Bluetooth air. There is no rate limiting in the HCI SCO sending.
Options for rate limiting: -
1) For best sound quality, the rate limiting should be based on the HCI hardware, and not any other source.
2) Only send hci sco when one receives an hci sco
This causes problems if received hci sco packets get dropped due to missed irqs etc. So it is better to not link the TX rate limiting to any RX packet rate.
3) Use the linux system time.
If the user changes the time, the linux system time get changed, so the rate limiting will be messed up each time the linux system time is changed. So, better not to use the linux system time.

So, we really want to use (1) if we can.
How about?: -

1) Each hci sco packet being send from the sco layer to the hci layer is tagged with a sequence number.
We send the hci sco packet from sco layer to hci layer.
When the tx_complete for that hci sco packet happens, the packet is returned to the sco layer being taged as complete and then the sco layer refills it with new data and sends it down again. As we tagged the hci sco packet with a sequence number, when it comes back we know which packet was actually send. As it has a sequence number on it, we can detect lost packets.
Because the completed packet is send back up to the sco layer, we are able to remove one malloc/free from the process.
Currently we have sco layer doing alloc, hci layer doing free.

2) Just use a limited sized queue.
sco layer fills the queue, but when the queue is full, it waits. the hci layer empties the queue as and when it needs to. Currently, I think we have a queue, but the size of the queue is not controllable from the sco layer.
It might be better if the hci layer reads the first item in the queue, schedules it for output. the hci layer only frees the item from the queue when it reaches tx_complete.
The tx_complete state is reached when the bluetooth hardware calls the interrupt handler.

I think option (2) fits more closely with the current bluez design. All I need is the answer to "How does one limit the queue size?".