A suggestion to solve the ALSA resampling problem.
Hi,
There are essentually 2 problems to solve. I will take the alsa code in the game doom3 as a way to demonstrate the problems.
doom3 uses a sample rate of 44100. Most sound cards now work at 48000 in hardware, so one needs to sample rate convert between 44100 to 48000 in software in order to get games like doom3 to play well. doom3 also opens the buffers at 1024 frames per period, and 4096 frames per buffer. It then writes 1024 frames at a time to the buffer. If the complete write fails, it complains and the sound sounds really bad. This approach is fine in the sound card can do periods of 1024 frames, but not all sound card hardware can, so we need to find an abstraction layer in order to deliver these period and buffer sizes to the application, while also serving the different period and buffer sizes of the hardware.
So, how about this.
The application can select any buffer size and period size they like. For simplicity, we might impose that number_of_periods is an integer.
So, the 44100 application buffer is 4096(buffer size),1024(period size) The hardware 48000 buffer is 2048(buffer size), 512(period size)
We add an intermediate buffer at 48000. This buffer size must be >= 4096*48000/44100. period size must be >= 1024*48000/44100. If we round up, we get:
intermediate buffer: 4458.23 -> 4459
intermediate period: 1114.56 -> 1115
For ease of transfer between the intermediate buffer and the hardware, we should really make the intermediate buffer_size = n*hardware period_size. So, we have the following equations to satify:intermediate_buffer_size = n * hardware_period_size. intermediate_buffer_size >= application_buffer*hardware_rate/application_rate. intermediate_period_size = hardware_period_size.
This results in the:intermediate_buffer_size = 4608 (9 * 512)intermediate_period_size = 512
Each time the sound card hardware interrupts (every 512 frames), alsa copies the next period from the intermediate buffer to the hardware buffer. This essentually allows for any sound card hardware to have as many periods as they wish, and thus as larger buffer size as they like. With the expense of one extra memcpy.
Each time the application writes to the application buffer, alsa-lib sample rate converts the number of frames in the snd_pcm_write() operation, and also writes those to the intermediate buffer.
Add to that some careful handling of position and avail pointers in each of the buffers. This should result in reliable sample rate/buffer size converted audio.
The next thing to consider is whether some applications can get sound to the pcm buffer without using snd_pcm_write(). For example memcpy for mmap applications. We would need some way to track these memcpy writes, so that the sample rate converter could take the new samples, and sample rate convert them into the intermediate buffer.
Does anyone have any ideas regarding this last point? If this last point never happens in alsa, then I will go ahead and start writing a patch for alsa to do all this.
James
-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
At Sat, 02 Jul 2005 13:23:42 +0100,
Quoted 39 lines · James Courtier-Dutton
Hi,
There are essentually 2 problems to solve. I will take the alsa code in the game doom3 as a way to demonstrate the problems.
doom3 uses a sample rate of 44100. Most sound cards now work at 48000 in hardware, so one needs to sample rate convert between 44100 to 48000 in software in order to get games like doom3 to play well. doom3 also opens the buffers at 1024 frames per period, and 4096 frames per buffer. It then writes 1024 frames at a time to the buffer. If the complete write fails, it complains and the sound sounds really bad. This approach is fine in the sound card can do periods of 1024 frames, but not all sound card hardware can, so we need to find an abstraction layer in order to deliver these period and buffer sizes to the application, while also serving the different period and buffer sizes of the hardware.
So, how about this.
The application can select any buffer size and period size they like. For simplicity, we might impose thatnumber_of_periodsis an integer.So, the 44100 application buffer is 4096(buffer size),1024(period size) The hardware 48000 buffer is 2048(buffer size), 512(period size)
We add an intermediate buffer at 48000. This buffer size must be >= 4096*48000/44100. period size must be >= 1024*48000/44100. If we round up, we get:
intermediate buffer: 4458.23 -> 4459
intermediate period: 1114.56 -> 1115For ease of transfer between the intermediate buffer and the hardware, we should really make the intermediate
buffer_size= n*hardwareperiod_size. So, we have the following equations to satify:intermediate_buffer_size= n *hardware_period_size.intermediate_buffer_size>= application_buffer*hardware_rate/application_rate.intermediate_period_size=hardware_period_size.
Quoted 3 lines
This results in the:
intermediate_buffer_size= 4608 (9 * 512)intermediate_period_size= 512
The sub-period is a good idea. However, this seems possible only when you can change the period size of the slave PCM arbitrarily. A typical bad example is dmix, which has the fixed period/buffer configuration.
Also, rounding up/down the fraction always introduces the round error. This may cause a drift in a long run.
Quoted 14 lines
Each time the sound card hardware interrupts (every 512 frames), alsa copies the next period from the intermediate buffer to the hardware buffer. This essentually allows for any sound card hardware to have as many periods as they wish, and thus as larger buffer size as they like. With the expense of one extra memcpy.
Each time the application writes to the application buffer, alsa-lib sample rate converts the number of frames in the
snd_pcm_write()operation, and also writes those to the intermediate buffer.Add to that some careful handling of position and avail pointers in each of the buffers. This should result in reliable sample rate/buffer size converted audio.
Quoted 5 lines
The next thing to consider is whether some applications can get sound to the pcm buffer without using
snd_pcm_write(). For example memcpy for mmap applications. We would need some way to track these memcpy writes, so that the sample rate converter could take the new samples, and sample rate convert them into the intermediate buffer.
Well, even snd_pcm_read/write() are done in the mmap mode in rate plugin...
The root of the problems of rate plugin is that we have no proper interrupts for the given period configuration for the master PCM. Using the irqs of the slave PCM may have restrictions that can't be solved easily as mentioned above. Hence, I feel there are only two solutions for this:
1. Allow variable period size
2. Introduce another interrupt source
1 will be a drastic change to the outside, so it's not an option right now although it can be a good extension in future.
2 means to extend the kernel side PCM handler to allow artibrary interrupt source(s). For example, suppose to create a new timer instance driven by the system timer. Then you can set any "tick" value for the certain timing.
The timer can be sync'ed with PCM DMA, too. That is, we adjust the internal tick resolution at each time PCM DMA irq is issued (a la PLL).
Also, for a better PCM emulation, we may introduce a new timer mode for tick scheduling. The timer instance has a queue of each wake-up duration. The user-space writes to the queue and reads in return. The size of queue corresponds to the number of periods of PCM buffer, and each queue value corresponds to the period size. You can set different values to adjust the rounding error.
It's just my $0.02.
Takashi
-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
Quoted 41 lines · Takashi Iwai
At Sat, 02 Jul 2005 13:23:42 +0100,
Quoted 39 lines · James Courtier-Dutton
Hi,
There are essentually 2 problems to solve. I will take the alsa code in the game doom3 as a way to demonstrate the problems.
doom3 uses a sample rate of 44100. Most sound cards now work at 48000 in hardware, so one needs to sample rate convert between 44100 to 48000 in software in order to get games like doom3 to play well. doom3 also opens the buffers at 1024 frames per period, and 4096 frames per buffer. It then writes 1024 frames at a time to the buffer. If the complete write fails, it complains and the sound sounds really bad. This approach is fine in the sound card can do periods of 1024 frames, but not all sound card hardware can, so we need to find an abstraction layer in order to deliver these period and buffer sizes to the application, while also serving the different period and buffer sizes of the hardware.
So, how about this.
The application can select any buffer size and period size they like. For simplicity, we might impose thatnumber_of_periodsis an integer.So, the 44100 application buffer is 4096(buffer size),1024(period size) The hardware 48000 buffer is 2048(buffer size), 512(period size)
We add an intermediate buffer at 48000. This buffer size must be >= 4096*48000/44100. period size must be >= 1024*48000/44100. If we round up, we get:
intermediate buffer: 4458.23 -> 4459
intermediate period: 1114.56 -> 1115For ease of transfer between the intermediate buffer and the hardware, we should really make the intermediate
buffer_size= n*hardwareperiod_size. So, we have the following equations to satify:intermediate_buffer_size= n *hardware_period_size.intermediate_buffer_size>= application_buffer*hardware_rate/application_rate.intermediate_period_size=hardware_period_size.
Quoted 8 lines
Quoted 3 lines
This results in the:
intermediate_buffer_size= 4608 (9 * 512)intermediate_period_size= 512The sub-period is a good idea. However, this seems possible only when you can change the period size of the slave PCM arbitrarily. A typical bad example is dmix, which has the fixed period/buffer configuration.
Which is the slave, and which is the master? master == hardware buffer, slave == application buffer or
master == application buffer, slave == hardware buffer.
The current method used in alsa-lib is to try to map the applications period directly onto the hardware period while changing sample rate at the same time. I am suggesting adding a new buffer in between the two, so that the application period writes into the intermediate buffer one application period at a time, and the intermediate buffer writes into the hardware buffer one hardware period at a time. So, there are 3 buffers here.
1) hardware buffer at 48000Hz.
2) intermediate buffer at 48000Hz.
3) Application buffer at 44100Hz.
The application buffer free runs and uses timer interrupts to trigger periods. The application buffer has it's own pointer so can tell the application the status of the buffer.
Each time the application writes to the application buffer, it fills the application buffer and updates the application hw pointers. It also runs the snd_pcm_write() through a sample rate converter, that results in a varying amount of frames written to the intermediate buffer.
The hardware buffer free runs and transfers data from the intermediate buffer each time the hardware period ellapses.
So, the application sees a constant period size, the hardware also sees a constant period size (but different from the application one). The intermediate buffer is used to buffer between the two.
Quoted 3 lines
Also, rounding up/down the fraction always introduces the round error. This may cause a drift in a long run.
The size of the intermediate buffer does not matter. It just has to be bigger than the application wants. The resampler in the snd_pcm_write() function will be using fractions, so the rounding of the buffer size is not a problem as there is no requirement to map one application period directly into one intermediate period.
Quoted 15 lines
Quoted 15 lines
Each time the sound card hardware interrupts (every 512 frames), alsa copies the next period from the intermediate buffer to the hardware buffer. This essentually allows for any sound card hardware to have as many periods as they wish, and thus as larger buffer size as they like. With the expense of one extra memcpy.
Each time the application writes to the application buffer, alsa-lib sample rate converts the number of frames in the
snd_pcm_write()operation, and also writes those to the intermediate buffer.Add to that some careful handling of position and avail pointers in each of the buffers. This should result in reliable sample rate/buffer size converted audio.
Quoted 8 lines
Quoted 5 lines
The next thing to consider is whether some applications can get sound to the pcm buffer without using
snd_pcm_write(). For example memcpy for mmap applications. We would need some way to track these memcpy writes, so that the sample rate converter could take the new samples, and sample rate convert them into the intermediate buffer.Well, even snd_pcm_read/write() are done in the mmap mode in rate plugin...
Good.
Quoted 8 lines
The root of the problems of rate plugin is that we have no proper interrupts for the given period configuration for the master PCM. Using the irqs of the slave PCM may have restrictions that can't be solved easily as mentioned above. Hence, I feel there are only two solutions for this:
Quoted 5 lines
1. Allow variable period size
2. Introduce another interrupt source1 will be a drastic change to the outside, so it's not an option right now although it can be a good extension in future.
I also think that this probably causes more problems than it helps with, even though I actually suggested it some time ago.
Quoted 5 lines
2 means to extend the kernel side PCM handler to allow artibrary interrupt source(s). For example, suppose to create a new timer instance driven by the system timer. Then you can set any "tick" value for the certain timing.
Some sound cards, like the Audigy and ca0106 have "wall clocks" that we can generate interrupts from, if we wish. This would be a good way to get interrupts arriving in sync with the sound card hardware. So, we would have 2 interrupts.
1) For period_elapsed() from the hardware buffer period interrupt. (The current interrupt)
2) For period_elapsed() from the timer for the intermediate period trigger. (From wall-clock, but I am not sure how many sound cards have that feature.
Quoted 4 lines
The timer can be sync'ed with PCM DMA, too. That is, we adjust the internal tick resolution at each time PCM DMA irq is issued (a la PLL).
Good method if the sound card hardware does not support "wall clock"
Quoted 9 lines
Also, for a better PCM emulation, we may introduce a new timer mode for tick scheduling. The timer instance has a queue of each wake-up duration. The user-space writes to the queue and reads in return. The size of queue corresponds to the number of periods of PCM buffer, and each queue value corresponds to the period size. You can set different values to adjust the rounding error.
Quoted 4 lines
It's just my $0.02.
Takashi
James
-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_idt77&alloc_id492&op=click
At Tue, 5 Jul 2005 15:46:13 +0100,
Quoted 47 lines · James Courtier-Dutton
Quoted 45 lines · Takashi Iwai
At Sat, 02 Jul 2005 13:23:42 +0100,
Quoted 42 lines · James Courtier-Dutton
Hi,
There are essentually 2 problems to solve. I will take the alsa code in the game doom3 as a way to demonstrate the problems.
doom3 uses a sample rate of 44100. Most sound cards now work at 48000 in hardware, so one needs to sample rate convert between 44100 to 48000 in software in order to get games like doom3 to play well. doom3 also opens the buffers at 1024 frames per period, and 4096 frames per buffer. It then writes 1024 frames at a time to the buffer. If the complete write fails, it complains and the sound sounds really bad. This approach is fine in the sound card can do periods of 1024 frames, but not all sound card hardware can, so we need to find an abstraction layer in order to deliver these period and buffer sizes to the application, while also serving the different period and buffer sizes of the hardware.
So, how about this.
The application can select any buffer size and period size they like. For simplicity, we might impose thatnumber_of_periodsis an integer.So, the 44100 application buffer is 4096(buffer size),1024(period size) The hardware 48000 buffer is 2048(buffer size), 512(period size)
We add an intermediate buffer at 48000. This buffer size must be >= 4096*48000/44100. period size must be >= 1024*48000/44100. If we round up, we get:
intermediate buffer: 4458.23 -> 4459
intermediate period: 1114.56 -> 1115For ease of transfer between the intermediate buffer and the hardware, we should really make the intermediate
buffer_size= n*hardwareperiod_size. So, we have the following equations to satify:intermediate_buffer_size= n *hardware_period_size.intermediate_buffer_size>= application_buffer*hardware_rate/application_rate.intermediate_period_size=hardware_period_size.This results in the:
intermediate_buffer_size= 4608 (9 * 512)intermediate_period_size= 512
Quoted 8 lines
Quoted 4 lines
The sub-period is a good idea. However, this seems possible only when you can change the period size of the slave PCM arbitrarily. A typical bad example is dmix, which has the fixed period/buffer configuration.
Which is the slave, and which is the master? master == hardware buffer, slave == application buffer or
master == application buffer, slave == hardware buffer.
In the alsa-lib sense, slave PCM = hardware buffer, master = application.
Quoted 11 lines
The current method used in alsa-lib is to try to map the applications period directly onto the hardware period while changing sample rate at the same time. I am suggesting adding a new buffer in between the two, so that the application period writes into the intermediate buffer one application period at a time, and the intermediate buffer writes into the hardware buffer one hardware period at a time. So, there are 3 buffers here.
1) hardware buffer at 48000Hz.
2) intermediate buffer at 48000Hz.
3) Application buffer at 44100Hz.
The application buffer free runs and uses timer interrupts to trigger periods.
Note that the current rate plugin already has an intermediate buffer. The samples are transferred in asynchrnous way.
The problem is that rate plugin prefers to transfer the chunk of data as a whole period. This limitation could be solved if we have arbitrary interrupt source.
Quoted 15 lines
The application buffer has it's own pointer so can tell the application the status of the buffer.
Each time the application writes to the application buffer, it fills the application buffer and updates the application hw pointers. It also runs thesnd_pcm_write()through a sample rate converter, that results in a varying amount of frames written to the intermediate buffer.
The hardware buffer free runs and transfers data from the intermediate buffer each time the hardware period ellapses.So, the application sees a constant period size, the hardware also sees a constant period size (but different from the application one). The intermediate buffer is used to buffer between the two.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
Also, rounding up/down the fraction always introduces the round error. This may cause a drift in a long run.
The size of the intermediate buffer does not matter. It just has to be bigger than the application wants. The resampler in the
snd_pcm_write()function will be using fractions, so the rounding of the buffer size is not a problem as there is no requirement to map one application period directly into one intermediate period.
The whole point here is the "period size", not the buffer size. Yes, the buffer size doesn't matter. But the period size does.
The period size defines the interrupt period, and ALSA assumes the constant period size on the hardware side. Since 44.1khz can't be aligned in integer to 48kHz, there must be a drift as long as you handle the constant integer period size (that is, if you take the irq only from the soundcard period boundary, as normal soundcards provide.)
Quoted 45 lines
Quoted 14 lines
Quoted 14 lines
Each time the sound card hardware interrupts (every 512 frames), alsa copies the next period from the intermediate buffer to the hardware buffer. This essentually allows for any sound card hardware to have as many periods as they wish, and thus as larger buffer size as they like. With the expense of one extra memcpy.
Each time the application writes to the application buffer, alsa-lib sample rate converts the number of frames in the
snd_pcm_write()operation, and also writes those to the intermediate buffer.Add to that some careful handling of position and avail pointers in each of the buffers. This should result in reliable sample rate/buffer size converted audio.
Quoted 8 lines
Quoted 5 lines
The next thing to consider is whether some applications can get sound to the pcm buffer without using
snd_pcm_write(). For example memcpy for mmap applications. We would need some way to track these memcpy writes, so that the sample rate converter could take the new samples, and sample rate convert them into the intermediate buffer.Well, even snd_pcm_read/write() are done in the mmap mode in rate plugin...
Good.
Quoted 8 lines
The root of the problems of rate plugin is that we have no proper interrupts for the given period configuration for the master PCM. Using the irqs of the slave PCM may have restrictions that can't be solved easily as mentioned above. Hence, I feel there are only two solutions for this:
Quoted 5 lines
1. Allow variable period size
2. Introduce another interrupt source1 will be a drastic change to the outside, so it's not an option right now although it can be a good extension in future.
I also think that this probably causes more problems than it helps with, even though I actually suggested it some time ago.
Quoted 5 lines
2 means to extend the kernel side PCM handler to allow artibrary interrupt source(s). For example, suppose to create a new timer instance driven by the system timer. Then you can set any "tick" value for the certain timing.
Quoted 9 lines
Some sound cards, like the Audigy and ca0106 have "wall clocks" that we can generate interrupts from, if we wish. This would be a good way to get interrupts arriving in sync with the sound card hardware. So, we would have 2 interrupts.
1) Forperiod_elapsed()from the hardware buffer period interrupt. (The current interrupt)
2) Forperiod_elapsed()from the timer for the intermediate period trigger. (From wall-clock, but I am not sure how many sound cards have that feature.
Remember that the interrupt source must be multiplex, too. For example, dmix would require an indepedent interrupt for each instance in a certain accuracy.
Takashi
-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click