Skip to contentSkip to conversation

Issue in alsa when dma complete race with pcm release

DISCUSSION Shengjiu Wang 2 participants 10 messages Jul 3, 2015 – Jul 15, 2015

lore
Share & tools
lore
#0 lore raw

Hi alsa-devel

There maybe a issue in ALSA when dma complete race with snd_pcm_release.
The pcm release and dma complete are in different thread. There is occasion that dmaengine_pcm_dma_complete() is called too late, some memory has been freed, the prtd is null. Then there is kernel dump.

Is there any solution for this issue? Thanks.

Best regards
wang shengjiu

#1 lore raw
Quoted 8 lines · Shengjiu Wang

Hi alsa-devel

There maybe a issue in ALSA when dma complete race with snd_pcm_release.
The pcm release and dma complete are in different thread. There is occasion that dmaengine_pcm_dma_complete() is called too late, some memory has been freed, the prtd is null. Then there is kernel dump.

Is there any solution for this issue? Thanks.

We need to introduce a synchronization primitive that allows a dmaengine client to synchronize to the execution of the complete callbacks.

terminate_all() unfortunately can't do this since terminate_all() might be called from within one of the complete callbacks and so would cause a deadlock if we'd wait for all complete callbacks to finish before terminate_all() returns.

So what is needed is a new function called dmaengine_sync() that will wait until all scheduled complete callbacks have finished. A call to this function needs to be put in snd_dmaengine_pcm_close() before the prtd is closed.

- Lars

#2 lore raw
Quoted 19 lines · Lars-Peter Clausen
Quoted 8 lines · Shengjiu Wang

Hi alsa-devel

There maybe a issue in ALSA when dma complete race with snd_pcm_release.
The pcm release and dma complete are in different thread. There is occasion that dmaengine_pcm_dma_complete() is called too late, some memory has been freed, the prtd is null. Then there is kernel dump.

Is there any solution for this issue? Thanks.

We need to introduce a synchronization primitive that allows a dmaengine client to synchronize to the execution of the complete callbacks.

terminate_all() unfortunately can't do this since terminate_all() might be called from within one of the complete callbacks and so would cause a deadlock if we'd wait for all complete callbacks to finish before terminate_all() returns.

Quoted 6 lines

So what is needed is a new function called dmaengine_sync() that will wait until all scheduled complete callbacks have finished. A call to this function needs to be put in snd_dmaengine_pcm_close() before the prtd is closed.

- Lars

How to check " all scheduled complete callbacks have finished"?

One concern is if add wait in snd_dmaengine_pcm_close(), which wil cause the snd_pcm_release is bound with dmaengine, when there is error in dma and no callback be called. Then the snd_pcm_release will not be released.

Best regards
wang shengjiu

#3 lore raw
Quoted 20 lines · Shengjiu Wang
Quoted 19 lines · Lars-Peter Clausen
Quoted 8 lines · Shengjiu Wang

Hi alsa-devel

There maybe a issue in ALSA when dma complete race with snd_pcm_release.
The pcm release and dma complete are in different thread. There is occasion that dmaengine_pcm_dma_complete() is called too late, some memory has been freed, the prtd is null. Then there is kernel dump.

Is there any solution for this issue? Thanks.

We need to introduce a synchronization primitive that allows a dmaengine client to synchronize to the execution of the complete callbacks.

terminate_all() unfortunately can't do this since terminate_all() might be called from within one of the complete callbacks and so would cause a deadlock if we'd wait for all complete callbacks to finish before terminate_all() returns.

Quoted 8 lines
Quoted 6 lines

So what is needed is a new function called dmaengine_sync() that will wait until all scheduled complete callbacks have finished. A call to this function needs to be put in snd_dmaengine_pcm_close() before the prtd is closed.

- Lars

How to check " all scheduled complete callbacks have finished"?

That will be up to the dmaengine driver. But it basically comes down to two things:

1) The driver needs to make sure that tasklet_schedule() is no longer called after terminate_all() has finished.
2) In the sync() callback call tasklet_kill() to make sure that it has finished running

Quoted 4 lines

One concern is if add wait in snd_dmaengine_pcm_close(), which wil cause the snd_pcm_release is bound with dmaengine, when there is error in dma and no callback be called. Then the snd_pcm_release will not be released.

The sync() function will only wait if there is a callback scheduled, if there is non scheduled it will return immediately.

#4 lore raw
Quoted 28 lines · Lars-Peter Clausen
Quoted 27 lines · Shengjiu Wang
Quoted 25 lines · Lars-Peter Clausen
Quoted 8 lines · Shengjiu Wang

Hi alsa-devel

There maybe a issue in ALSA when dma complete race with snd_pcm_release.
The pcm release and dma complete are in different thread. There is occasion that dmaengine_pcm_dma_complete() is called too late, some memory has been freed, the prtd is null. Then there is kernel dump.

Is there any solution for this issue? Thanks.

We need to introduce a synchronization primitive that allows a dmaengine client to synchronize to the execution of the complete callbacks.

terminate_all() unfortunately can't do this since terminate_all() might be called from within one of the complete callbacks and so would cause a deadlock if we'd wait for all complete callbacks to finish before terminate_all() returns.

So what is needed is a new function called dmaengine_sync() that will wait until all scheduled complete callbacks have finished. A call to this function needs to be put in snd_dmaengine_pcm_close() before the prtd is closed.

- Lars

Quoted 7 lines
Quoted 1 line

How to check " all scheduled complete callbacks have finished"?

That will be up to the dmaengine driver. But it basically comes down to two things:

1) The driver needs to make sure that tasklet_schedule() is no longer called after terminate_all() has finished.

Some driver can't make sure this. The dma interrupt may come later after terminate_all.

Quoted 10 lines

2) In the sync() callback call tasklet_kill() to make sure that it has finished running

Quoted 4 lines

One concern is if add wait in snd_dmaengine_pcm_close(), which wil cause the snd_pcm_release is bound with dmaengine, when there is error in dma and no callback be called. Then the snd_pcm_release will not be released.

The sync() function will only wait if there is a callback scheduled, if there is non scheduled it will return immediately.

Do you have other method to resolve this issue? or simple method?

Best regards
wang shengjiu

#5 lore raw
Quoted 31 lines · Shengjiu Wang
Quoted 30 lines · Lars-Peter Clausen
Quoted 28 lines · Shengjiu Wang
Quoted 19 lines · Lars-Peter Clausen
Quoted 8 lines · Shengjiu Wang

Hi alsa-devel

There maybe a issue in ALSA when dma complete race with snd_pcm_release.
The pcm release and dma complete are in different thread. There is occasion that dmaengine_pcm_dma_complete() is called too late, some memory has been freed, the prtd is null. Then there is kernel dump.

Is there any solution for this issue? Thanks.

We need to introduce a synchronization primitive that allows a dmaengine client to synchronize to the execution of the complete callbacks.

terminate_all() unfortunately can't do this since terminate_all() might be called from within one of the complete callbacks and so would cause a deadlock if we'd wait for all complete callbacks to finish before terminate_all() returns.

Quoted 6 lines

So what is needed is a new function called dmaengine_sync() that will wait until all scheduled complete callbacks have finished. A call to this function needs to be put in snd_dmaengine_pcm_close() before the prtd is closed.

- Lars

How to check " all scheduled complete callbacks have finished"?

Quoted 7 lines
Quoted 5 lines

That will be up to the dmaengine driver. But it basically comes down to two things:

1) The driver needs to make sure that tasklet_schedule() is no longer called after terminate_all() has finished.

Some driver can't make sure this. The dma interrupt may come later after terminate_all.

Most drivers can't make sure that the interrupt routine is not executed after terminate_all() has been called, since both are fully asynchronous. But what the driver needs to take care of is to synchronize the two e.g. using a spin_lock() and make sure that if terminate_all() has been called tasklet_schedule() is not called, even if the isr is executed. Most drivers get this rigght.

Quoted 8 lines
Quoted 8 lines

2) In the sync() callback call tasklet_kill() to make sure that it has finished running

Quoted 4 lines

One concern is if add wait in snd_dmaengine_pcm_close(), which wil cause the snd_pcm_release is bound with dmaengine, when there is error in dma and no callback be called. Then the snd_pcm_release will not be released.

Quoted 5 lines
Quoted 2 lines

The sync() function will only wait if there is a callback scheduled, if there is non scheduled it will return immediately.

Do you have other method to resolve this issue? or simple method?

No, this is the way to go. You have two asynchronous processes and you want to get deterministic execution ordering between the two you need to add a synchronization primitive.

- Lars

#6 lore raw
Quoted 59 lines · Lars-Peter Clausen
Quoted 31 lines · Shengjiu Wang
Quoted 30 lines · Lars-Peter Clausen
Quoted 28 lines · Shengjiu Wang
Quoted 19 lines · Lars-Peter Clausen
Quoted 8 lines · Shengjiu Wang

Hi alsa-devel

There maybe a issue in ALSA when dma complete race with snd_pcm_release.
The pcm release and dma complete are in different thread. There is occasion that dmaengine_pcm_dma_complete() is called too late, some memory has been freed, the prtd is null. Then there is kernel dump.

Is there any solution for this issue? Thanks.

We need to introduce a synchronization primitive that allows a dmaengine client to synchronize to the execution of the complete callbacks.

terminate_all() unfortunately can't do this since terminate_all() might be called from within one of the complete callbacks and so would cause a deadlock if we'd wait for all complete callbacks to finish before terminate_all() returns.

Quoted 6 lines

So what is needed is a new function called dmaengine_sync() that will wait until all scheduled complete callbacks have finished. A call to this function needs to be put in snd_dmaengine_pcm_close() before the prtd is closed.

- Lars

How to check " all scheduled complete callbacks have finished"?

Quoted 7 lines
Quoted 5 lines

That will be up to the dmaengine driver. But it basically comes down to two things:

1) The driver needs to make sure that tasklet_schedule() is no longer called after terminate_all() has finished.

Some driver can't make sure this. The dma interrupt may come later after terminate_all.

Most drivers can't make sure that the interrupt routine is not executed after terminate_all() has been called, since both are fully asynchronous. But what the driver needs to take care of is to synchronize the two e.g. using a spin_lock() and make sure that if terminate_all() has been called tasklet_schedule() is not called, even if the isr is executed. Most drivers get this rigght.

Quoted 12 lines
Quoted 10 lines

2) In the sync() callback call tasklet_kill() to make sure that it has finished running

Quoted 4 lines

One concern is if add wait in snd_dmaengine_pcm_close(), which wil cause the snd_pcm_release is bound with dmaengine, when there is error in dma and no callback be called. Then the snd_pcm_release will not be released.

The sync() function will only wait if there is a callback scheduled, if there is non scheduled it will return immediately.

Quoted 8 lines
Quoted 1 line

Do you have other method to resolve this issue? or simple method?

No, this is the way to go. You have two asynchronous processes and you want to get deterministic execution ordering between the two you need to add a synchronization primitive.

- Lars

Thanks your advice. It is workable. But need to add new api in dmaengine.h

I think about another method, can we add spin_lock in dmaengine_pcm_dma_complete() and snd_dmaengine_pcm_close()? and add a flag for pcm_close, if pcm closed, then just do nothing in dma complete. How do you think for this?

best regards
wang shengjiu

#7 lore raw
Quoted 66 lines · Shengjiu Wang
Quoted 65 lines · Lars-Peter Clausen
Quoted 31 lines · Shengjiu Wang
Quoted 30 lines · Lars-Peter Clausen
Quoted 28 lines · Shengjiu Wang
Quoted 19 lines · Lars-Peter Clausen
Quoted 8 lines · Shengjiu Wang

Hi alsa-devel

There maybe a issue in ALSA when dma complete race with snd_pcm_release.
The pcm release and dma complete are in different thread. There is occasion that dmaengine_pcm_dma_complete() is called too late, some memory has been freed, the prtd is null. Then there is kernel dump.

Is there any solution for this issue? Thanks.

We need to introduce a synchronization primitive that allows a dmaengine client to synchronize to the execution of the complete callbacks.

terminate_all() unfortunately can't do this since terminate_all() might be called from within one of the complete callbacks and so would cause a deadlock if we'd wait for all complete callbacks to finish before terminate_all() returns.

Quoted 6 lines

So what is needed is a new function called dmaengine_sync() that will wait until all scheduled complete callbacks have finished. A call to this function needs to be put in snd_dmaengine_pcm_close() before the prtd is closed.

- Lars

How to check " all scheduled complete callbacks have finished"?

Quoted 7 lines
Quoted 5 lines

That will be up to the dmaengine driver. But it basically comes down to two things:

1) The driver needs to make sure that tasklet_schedule() is no longer called after terminate_all() has finished.

Some driver can't make sure this. The dma interrupt may come later after terminate_all.

Most drivers can't make sure that the interrupt routine is not executed after terminate_all() has been called, since both are fully asynchronous. But what the driver needs to take care of is to synchronize the two e.g. using a spin_lock() and make sure that if terminate_all() has been called tasklet_schedule() is not called, even if the isr is executed. Most drivers get this rigght.

Quoted 8 lines
Quoted 8 lines

2) In the sync() callback call tasklet_kill() to make sure that it has finished running

Quoted 4 lines

One concern is if add wait in snd_dmaengine_pcm_close(), which wil cause the snd_pcm_release is bound with dmaengine, when there is error in dma and no callback be called. Then the snd_pcm_release will not be released.

Quoted 5 lines
Quoted 2 lines

The sync() function will only wait if there is a callback scheduled, if there is non scheduled it will return immediately.

Do you have other method to resolve this issue? or simple method?

No, this is the way to go. You have two asynchronous processes and you want to get deterministic execution ordering between the two you need to add a synchronization primitive.

Quoted 8 lines
Quoted 2 lines

- Lars

Thanks your advice. It is workable. But need to add new api in dmaengine.h

I think about another method, can we add spin_lock in dmaengine_pcm_dma_complete() and snd_dmaengine_pcm_close()? and add a flag for pcm_close, if pcm closed, then just do nothing in dma complete. How do you think for this?

Since you are freeing the memory that would store the flag that wont work.

- Lars

#8 lore raw
Quoted 71 lines · Lars-Peter Clausen
Quoted 70 lines · Shengjiu Wang
Quoted 59 lines · Lars-Peter Clausen
Quoted 31 lines · Shengjiu Wang
Quoted 30 lines · Lars-Peter Clausen
Quoted 28 lines · Shengjiu Wang
Quoted 19 lines · Lars-Peter Clausen
Quoted 8 lines · Shengjiu Wang

Hi alsa-devel

There maybe a issue in ALSA when dma complete race with snd_pcm_release.
The pcm release and dma complete are in different thread. There is occasion that dmaengine_pcm_dma_complete() is called too late, some memory has been freed, the prtd is null. Then there is kernel dump.

Is there any solution for this issue? Thanks.

We need to introduce a synchronization primitive that allows a dmaengine client to synchronize to the execution of the complete callbacks.

terminate_all() unfortunately can't do this since terminate_all() might be called from within one of the complete callbacks and so would cause a deadlock if we'd wait for all complete callbacks to finish before terminate_all() returns.

Quoted 6 lines

So what is needed is a new function called dmaengine_sync() that will wait until all scheduled complete callbacks have finished. A call to this function needs to be put in snd_dmaengine_pcm_close() before the prtd is closed.

- Lars

How to check " all scheduled complete callbacks have finished"?

Quoted 7 lines
Quoted 5 lines

That will be up to the dmaengine driver. But it basically comes down to two things:

1) The driver needs to make sure that tasklet_schedule() is no longer called after terminate_all() has finished.

Some driver can't make sure this. The dma interrupt may come later after terminate_all.

Most drivers can't make sure that the interrupt routine is not executed after terminate_all() has been called, since both are fully asynchronous. But what the driver needs to take care of is to synchronize the two e.g. using a spin_lock() and make sure that if terminate_all() has been called tasklet_schedule() is not called, even if the isr is executed. Most drivers get this rigght.

Quoted 12 lines
Quoted 10 lines

2) In the sync() callback call tasklet_kill() to make sure that it has finished running

Quoted 4 lines

One concern is if add wait in snd_dmaengine_pcm_close(), which wil cause the snd_pcm_release is bound with dmaengine, when there is error in dma and no callback be called. Then the snd_pcm_release will not be released.

The sync() function will only wait if there is a callback scheduled, if there is non scheduled it will return immediately.

Quoted 8 lines
Quoted 1 line

Do you have other method to resolve this issue? or simple method?

No, this is the way to go. You have two asynchronous processes and you want to get deterministic execution ordering between the two you need to add a synchronization primitive.

- Lars

Thanks your advice. It is workable. But need to add new api in dmaengine.h

Quoted 9 lines
Quoted 4 lines

I think about another method, can we add spin_lock in dmaengine_pcm_dma_complete() and snd_dmaengine_pcm_close()? and add a flag for pcm_close, if pcm closed, then just do nothing in dma complete. How do you think for this?

Since you are freeing the memory that would store the flag that wont work.

- Lars

It seems the snd_pcm_substream is not released, can we add flag in this struct?\

best regards
wang shengjiu

#9 lore raw
Quoted 77 lines · Shengjiu Wang
Quoted 76 lines · Lars-Peter Clausen
Quoted 74 lines · Shengjiu Wang
Quoted 59 lines · Lars-Peter Clausen
Quoted 31 lines · Shengjiu Wang
Quoted 30 lines · Lars-Peter Clausen
Quoted 28 lines · Shengjiu Wang
Quoted 19 lines · Lars-Peter Clausen
Quoted 8 lines · Shengjiu Wang

Hi alsa-devel

There maybe a issue in ALSA when dma complete race with snd_pcm_release.
The pcm release and dma complete are in different thread. There is occasion that dmaengine_pcm_dma_complete() is called too late, some memory has been freed, the prtd is null. Then there is kernel dump.

Is there any solution for this issue? Thanks.

We need to introduce a synchronization primitive that allows a dmaengine client to synchronize to the execution of the complete callbacks.

terminate_all() unfortunately can't do this since terminate_all() might be called from within one of the complete callbacks and so would cause a deadlock if we'd wait for all complete callbacks to finish before terminate_all() returns.

Quoted 6 lines

So what is needed is a new function called dmaengine_sync() that will wait until all scheduled complete callbacks have finished. A call to this function needs to be put in snd_dmaengine_pcm_close() before the prtd is closed.

- Lars

How to check " all scheduled complete callbacks have finished"?

Quoted 7 lines
Quoted 5 lines

That will be up to the dmaengine driver. But it basically comes down to two things:

1) The driver needs to make sure that tasklet_schedule() is no longer called after terminate_all() has finished.

Some driver can't make sure this. The dma interrupt may come later after terminate_all.

Most drivers can't make sure that the interrupt routine is not executed after terminate_all() has been called, since both are fully asynchronous. But what the driver needs to take care of is to synchronize the two e.g. using a spin_lock() and make sure that if terminate_all() has been called tasklet_schedule() is not called, even if the isr is executed. Most drivers get this rigght.

Quoted 12 lines
Quoted 10 lines

2) In the sync() callback call tasklet_kill() to make sure that it has finished running

Quoted 4 lines

One concern is if add wait in snd_dmaengine_pcm_close(), which wil cause the snd_pcm_release is bound with dmaengine, when there is error in dma and no callback be called. Then the snd_pcm_release will not be released.

The sync() function will only wait if there is a callback scheduled, if there is non scheduled it will return immediately.

Quoted 8 lines
Quoted 1 line

Do you have other method to resolve this issue? or simple method?

No, this is the way to go. You have two asynchronous processes and you want to get deterministic execution ordering between the two you need to add a synchronization primitive.

- Lars

Thanks your advice. It is workable. But need to add new api in dmaengine.h

I think about another method, can we add spin_lock in dmaengine_pcm_dma_complete() and snd_dmaengine_pcm_close()? and add a flag for pcm_close, if pcm closed, then just do nothing in dma complete. How do you think for this?

Quoted 5 lines
Quoted 4 lines

Since you are freeing the memory that would store the flag that wont work.

- Lars

It seems the snd_pcm_substream is not released, can we add flag in this struct?\

No, there is no guarantee that the substream hasn't been freed either.