compress: add support for gapless playback
Subsystems (3)
RFC 1/1 compress: add support for gapless playback
From: Jeeja KP <[email protected]>
this add new API for sound compress to support gapless playback. As noted in Documentation change, we add API to send metadata of encoder and padding delay to DSP. Also add API for indicating EOF and switching to subsequent track
Also bump the compress API version
--- Documentation/sound/alsa/compress_offload.txt | 22 +++++++++++ include/sound/compress_driver.h | 2 + include/uapi/sound/compress_offload.h | 18 ++++++++- sound/core/compress_offload.c | 51 +++++++++++++++++++++++++ 4 files changed, 92 insertions(+), 1 deletions(-)
Documentation/sound/alsa/compress_offload.txt +22 -0
@@ -145,6 +145,28 @@ Modifications include: - Addition of encoding options when required (derived from OpenMAX IL) - Addition of rateControlSupported (missing in OpenMAX AL) +Gapless Playback +================ +When playing thru an album, the decoders have the ability to skip the encoder +delay and padding and directly move from one track content to another. The end +user can perceive this as gapless playback as we dont have silence while +switching from one track to another + +The decoder needs to know the encoder delay and encoder padding. So we need to +pass this to DSP. Also DSP and userspace needs to switch from one track to +another and start using data for second track. + +The main additions are: + +- set_metadata +This routine sets the encoder delay and encoder padding. This can be used by +decoder to strip the silence + +- partial drain +This is called when end of file is reached. The userspace can inform DSP that +EOF is reached and now DSP can start skipping padding delay. Also next write +data would belong to next track + Not supported: - Support for VoIP/circuit-switched calls is not the target of this
include/sound/compress_driver.h +2 -0
@@ -110,6 +110,8 @@ struct snd_compr_ops { struct snd_compr_params *params);int (*get_params)(struct snd_compr_stream *stream,struct snd_codec *params); +int (*set_metadata)(struct snd_compr_stream *stream, +struct snd_compr_metadata *metadata);int (*trigger)(struct snd_compr_stream *stream,int cmd);int (*pointer)(struct snd_compr_stream *stream,struct snd_compr_tstamp *tstamp);
include/uapi/sound/compress_offload.h +17 -1
@@ -30,7 +30,7 @@#include <sound/compress_params.h> -#define SNDRV_COMPRESS_VERSION SNDRV_PROTOCOL_VERSION(0, 1, 0) +#define SNDRV_COMPRESS_VERSION SNDRV_PROTOCOL_VERSION(0, 1, 1)/** * struct snd_compressed_buffer: compressed buffer * @fragment_size: size of buffer fragment in bytes
@@ -122,6 +122,18 @@ struct snd_compr_codec_caps { };/** +* struct snd_compr_metadata: compressed stream metadata +* @encoder_delay: no of samples inserted by the encoder at the beginning +* of the track +* @encoder_padding: no of samples appended by the encoder at the end +* of the track +*/ +struct snd_compr_metadata { +__u32 encoder_delay; +__u32 encoder_padding; +}; + +/** * compress path ioctl definitions * SNDRV_COMPRESS_GET_CAPS: Query capability of DSP * SNDRV_COMPRESS_GET_CODEC_CAPS: Query capability of a codec
@@ -145,6 +157,8 @@ struct snd_compr_codec_caps { struct snd_compr_codec_caps)#define SNDRV_COMPRESS_SET_PARAMS _IOW('C', 0x12, struct snd_compr_params)#define SNDRV_COMPRESS_GET_PARAMS _IOR('C', 0x13, struct snd_codec) +#define SNDRV_COMPRESS_SET_METADATA _IOW('C', 0x14,\ + struct snd_compr_metadata)#define SNDRV_COMPRESS_TSTAMP _IOR('C', 0x20, struct snd_compr_tstamp)#define SNDRV_COMPRESS_AVAIL _IOR('C', 0x21, struct snd_compr_avail)#define SNDRV_COMPRESS_PAUSE _IO('C', 0x30)
@@ -152,10 +166,12 @@ struct snd_compr_codec_caps {#define SNDRV_COMPRESS_START _IO('C', 0x32)#define SNDRV_COMPRESS_STOP _IO('C', 0x33)#define SNDRV_COMPRESS_DRAIN _IO('C', 0x34) +#define SNDRV_COMPRESS_PARTIAL_DRAIN _IO('C', 0x35)/* * TODO * 1. add mmap support * */ #define SND_COMPR_TRIGGER_DRAIN 7/*FIXME move this to pcm.h */ +#define SND_COMPR_TRIGGER_PARTIAL_DRAIN 8#endif
sound/core/compress_offload.c +51 -0
75 lines not on this page · raw message
Signature · 1 line
1.7.0.4
Documentation/sound/alsa/compress_offload.txt Modifications include: · line 153
Quoted 7 lines
Documentation/sound/alsa/compress_offload.txt +6
-0+Gapless Playback +================ +When playing thru an album, the decoders have the ability to skip the encoder +delay and padding and directly move from one track content to another. The end +user can perceive this as gapless playback as we dont have silence while +switching from one track to another
You want to clarify that there might be low-intensity noises due to encoding. Perfect gapless is difficult to reach with all types of compressed data, but works fine with most music content. Also you want to mention that this metadata is extracted from ID3/MP4 headers and are not present by default in the bitstream, hence the need for a new interface to pass this information to the DSP.
Documentation/sound/alsa/compress_offload.txt Modifications include: · line 163
Quoted 3 lines
+-
set_metadataDocumentation/sound/alsa/compress_offload.txt +2
-0+This routine sets the encoder delay and encoder padding. This can be used by +decoder to strip the silence
You did not specify that this needs to be set before the first write of the next track.
Documentation/sound/alsa/compress_offload.txt Modifications include: · line 168
Quoted 5 lines
Documentation/sound/alsa/compress_offload.txt +5
-0+ +- partial drain +This is called when end of file is reached. The userspace can inform DSP that +EOF is reached and now DSP can start skipping padding delay. Also next write +data would belong to next track
DSP skips padding delay after decoding all samples in the track, not as soon as EOF is reached. A sequence as seen from userspace might be useful here to avoid random interpretations of the API semantics.
include/uapi/sound/compress_offload.h struct snd_compr_codec_caps { · line 134
Quoted 11 lines
/**
+ * structsnd_compr_metadata: compressed stream metadata
+ * @encoder_delay: no of samples inserted by the encoder at the beginning
+ * of the track
+ * @encoder_padding: no of samples appended by the encoder at the end
+ * of the track
+ */include/uapi/sound/compress_offload.h +4
-0+struct snd_compr_metadata { +__u32 encoder_delay; +__u32 encoder_padding; +};
You need to pad this structure with reserved bytes for future evolutions. Things like ReplayGain, etc, might be of interest for some implementations. Let's not be cheap here
sound/core/compress_offload.c out: · line 524
Quoted 8 lines
sound/core/compress_offload.c +8
-0+static int +snd_compr_set_metadata (struct snd_compr_stream *stream,unsigned long arg) +{ +struct snd_compr_metadata *metadata; +int retval; + +if (!stream->ops->set_metadata) +return -ENXIO ;
Is this really a fatal error? Or do we want to mandate that gapless be supported by all implementations?
Quoted 7 lines · Pierre-Louis Bossart
Quoted 7 lines
/**
+ * structsnd_compr_metadata: compressed stream metadata
+ * @encoder_delay: no of samples inserted by the encoder at the beginning
+ * of the track
+ * @encoder_padding: no of samples appended by the encoder at the end
+ * of the track
+ */
Quoted 8 lines
include/uapi/sound/compress_offload.h struct snd_compr_codec_caps { · line 134
Quoted 4 lines
include/uapi/sound/compress_offload.h +4
-0+struct snd_compr_metadata { +__u32 encoder_delay; +__u32 encoder_padding; +};You need to pad this structure with reserved bytes for future evolutions. Things like ReplayGain, etc, might be of interest for some implementations. Let's not be cheap here
Ok, makes sense, will update this and documentation.
Quoted 11 lines
sound/core/compress_offload.c out: · line 524
Quoted 8 lines
sound/core/compress_offload.c +8
-0+static int +snd_compr_set_metadata (struct snd_compr_stream *stream,unsigned long arg) +{ +struct snd_compr_metadata *metadata; +int retval; + +if (!stream->ops->set_metadata) +return -ENXIO ;Is this really a fatal error? Or do we want to mandate that gapless be supported by all implementations?
Fatal err? if DSP doesnt support metadata callback then we are reporting error ENXIO to userpsace. No we shouldnt mandate, its upto DSP folks to see what they should and can support.
--
~Vinod
Quoted 13 lines · Vinod Koul
Quoted 10 lines · Pierre-Louis Bossart
sound/core/compress_offload.c out: · line 524
Quoted 7 lines
sound/core/compress_offload.c +7
-0+snd_compr_set_metadata(struct snd_compr_stream *stream, unsigned long arg) +{ +struct snd_compr_metadata *metadata; +int retval; + +if (!stream->ops->set_metadata) +return -ENXIO ;Is this really a fatal error? Or do we want to mandate that gapless be supported by all implementations?
Quoted 3 lines
Fatal err? if DSP doesnt support metadata callback then we are reporting error ENXIO to userpsace. No we shouldnt mandate, its upto DSP folks to see what they should and can support.
Well, up to userspace to see how it handles the error anyway. For example an application may want to fall back to PCM playback with gapless done on the CPU if the driver doesn't do it. But then given that there's more than one piece of metadata in the struct perhaps we need some way for the application to figure out what's supported?
Quoted 11 lines
Quoted 10 lines
Quoted 6 lines
sound/core/compress_offload.c out: · line 524
Quoted 3 lines
sound/core/compress_offload.c +2
-0+if (!stream->ops->set_metadata) +return -ENXIO ;Is this really a fatal error? Or do we want to mandate that gapless be supported by all implementations?
Fatal err? if DSP doesnt support metadata callback then we are reporting error ENXIO to userpsace. No we shouldnt mandate, its upto DSP folks to see what they should and can support.
Quoted 5 lines
Well, up to userspace to see how it handles the error anyway. For example an application may want to fall back to PCM playback with gapless done on the CPU if the driver doesn't do it. But then given that there's more than one piece of metadata in the struct perhaps we need some way for the application to figure out what's supported?
Maybe we should expose what's supported as part of the decoder query process, i.e. expose support for metadata as part of the decoder capabilities rather than as part of the set_metadata part? Error checks after inits are painful. we could have a "metadata_support" bit-field that the application could check.
Quoted 17 lines · Pierre-Louis Bossart
Quoted 11 lines
Quoted 10 lines
Quoted 6 lines
sound/core/compress_offload.c out: · line 524
Quoted 3 lines
sound/core/compress_offload.c +2
-0+if (!stream->ops->set_metadata) +return -ENXIO ;Is this really a fatal error? Or do we want to mandate that gapless be supported by all implementations?
Fatal err? if DSP doesnt support metadata callback then we are reporting error ENXIO to userpsace. No we shouldnt mandate, its upto DSP folks to see what they should and can support.
Quoted 5 lines
Well, up to userspace to see how it handles the error anyway. For example an application may want to fall back to PCM playback with gapless done on the CPU if the driver doesn't do it. But then given that there's more than one piece of metadata in the struct perhaps we need some way for the application to figure out what's supported?
Quoted 5 lines
Maybe we should expose what's supported as part of the decoder query process, i.e. expose support for metadata as part of the decoder capabilities rather than as part of the
set_metadatapart? Error checks after inits are painful. we could have a "metadata_support" bit-field that the application could check.
Well I agree that would be a smarter way to handle. Make one of the reserved fields in decoder caps for metadata support.
--
~Vinod
Quoted 18 lines · Mark Brown
Quoted 13 lines · Vinod Koul
Quoted 10 lines · Pierre-Louis Bossart
sound/core/compress_offload.c out: · line 524
Quoted 7 lines
sound/core/compress_offload.c +7
-0+snd_compr_set_metadata(struct snd_compr_stream *stream, unsigned long arg) +{ +struct snd_compr_metadata *metadata; +int retval; + +if (!stream->ops->set_metadata) +return -ENXIO ;Is this really a fatal error? Or do we want to mandate that gapless be supported by all implementations?
Quoted 3 lines
Fatal err? if DSP doesnt support metadata callback then we are reporting error ENXIO to userpsace. No we shouldnt mandate, its upto DSP folks to see what they should and can support.
Quoted 5 lines
Well, up to userspace to see how it handles the error anyway. For example an application may want to fall back to PCM playback with gapless done on the CPU if the driver doesn't do it. But then given that there's more than one piece of metadata in the struct perhaps we need some way for the application to figure out what's supported?
This return would mean that API is not supported. If there is some other error wrt metadata set then driver needs to send appropriate error which can help userland figure out what caused this error
--
~Vinod
At Tue, 05 Feb 2013 20:44:42 -0600,
Quoted 16 lines · Pierre-Louis Bossart
include/uapi/sound/compress_offload.h struct snd_compr_codec_caps { · line 134
Quoted 12 lines
/**
+ * structsnd_compr_metadata: compressed stream metadata
+ * @encoder_delay: no of samples inserted by the encoder at the beginning
+ * of the track
+ * @encoder_padding: no of samples appended by the encoder at the end
+ * of the track
+ */include/uapi/sound/compress_offload.h +4
-0+struct snd_compr_metadata { +__u32 encoder_delay; +__u32 encoder_padding; +};You need to pad this structure with reserved bytes for future evolutions. Things like ReplayGain, etc, might be of interest for some implementations. Let's not be cheap here
Or, make it a single key/value. User-space can call the ioctl multiple times with different key/value pairs.
(BTW, don't you need get_metadata? IMO, it was a mistake that we
didn't provide get_* variants for PCM parameters.)
thanks,
Takashi
Quoted 19 lines · Takashi Iwai
At Tue, 05 Feb 2013 20:44:42 -0600,
Quoted 16 lines · Pierre-Louis Bossart
include/uapi/sound/compress_offload.h struct snd_compr_codec_caps { · line 134
Quoted 12 lines
/**
+ * structsnd_compr_metadata: compressed stream metadata
+ * @encoder_delay: no of samples inserted by the encoder at the beginning
+ * of the track
+ * @encoder_padding: no of samples appended by the encoder at the end
+ * of the track
+ */include/uapi/sound/compress_offload.h +4
-0+struct snd_compr_metadata { +__u32 encoder_delay; +__u32 encoder_padding; +};You need to pad this structure with reserved bytes for future evolutions. Things like ReplayGain, etc, might be of interest for some implementations. Let's not be cheap here
Quoted 2 lines
Or, make it a single key/value. User-space can call the ioctl multiple times with different key/value pairs.
Yes that would be smarter from ABI POV. But do we really have many params. And also in this case setting too many varible one by one can be time consuming....
Quoted 3 lines
(BTW, don't you need
get_metadata? IMO, it was a mistake that we
didn't provide get_* variants for PCM parameters.)
For above case no, as metadata is from file parser and send to decoder. But yes when we are encoding this will help. I think I will add that as well
--
~Vinod
At Wed, 6 Feb 2013 06:02:29 -0800,
Quoted 21 lines · Vinod Koul
Quoted 19 lines · Takashi Iwai
At Tue, 05 Feb 2013 20:44:42 -0600,
Quoted 16 lines · Pierre-Louis Bossart
include/uapi/sound/compress_offload.h struct snd_compr_codec_caps { · line 134
Quoted 12 lines
/**
+ * structsnd_compr_metadata: compressed stream metadata
+ * @encoder_delay: no of samples inserted by the encoder at the beginning
+ * of the track
+ * @encoder_padding: no of samples appended by the encoder at the end
+ * of the track
+ */include/uapi/sound/compress_offload.h +4
-0+struct snd_compr_metadata { +__u32 encoder_delay; +__u32 encoder_padding; +};You need to pad this structure with reserved bytes for future evolutions. Things like ReplayGain, etc, might be of interest for some implementations. Let's not be cheap here
Quoted 4 lines
Quoted 2 lines
Or, make it a single key/value. User-space can call the ioctl multiple times with different key/value pairs.
Yes that would be smarter from ABI POV. But do we really have many params. And also in this case setting too many varible one by one can be time consuming....
Would it really matter? The ioctl itself is cheap and fast. And, with the fixed metadata key/value pair, you don't need a kmalloc. It's known to be small to fit on stack.
Of course, if DSP's response to each parameter change is slow, it's a problem. Or if DSP requires the updates of multiple parameters at the same time, the key/value model doesn't work, too.
Takashi
At Tue, 5 Feb 2013 06:21:25 -0800,
Quoted 7 lines · Vinod Koul
From: Jeeja KP <[email protected]>
this add new API for sound compress to support gapless playback. As noted in Documentation change, we add API to send metadata of encoder and padding delay to DSP. Also add API for indicating EOF and switching to subsequent track
I can understand that the metadata is somehow handled in DSP for seamless switching, but the operation with partial drain is a bit unclear. How is the operation flow? Does it drain until which point? In your patch, the runtime state is changed to SETUP after the partial drain, so the app is requested to give START again?
thanks,
Takashi
Quoted 198 lines
Also bump the compress API version
Signed-off-by Jeeja KP <[email protected]>Signed-off-by Vinod Koul <[email protected]>--- Documentation/sound/alsa/compress_offload.txt | 22 +++++++++++ include/sound/compress_driver.h | 2 + include/uapi/sound/compress_offload.h | 18 ++++++++- sound/core/compress_offload.c | 51 +++++++++++++++++++++++++ 4 files changed, 92 insertions(+), 1 deletions(-)Documentation/sound/alsa/compress_offload.txt +22
-0@@ -145,6 +145,28 @@ Modifications include: - Addition of encoding options when required (derived from OpenMAX IL) - Addition of rateControlSupported (missing in OpenMAX AL) +Gapless Playback +================ +When playing thru an album, the decoders have the ability to skip the encoder +delay and padding and directly move from one track content to another. The end +user can perceive this as gapless playback as we dont have silence while +switching from one track to another + +The decoder needs to know the encoder delay and encoder padding. So we need to +pass this to DSP. Also DSP and userspace needs to switch from one track to +another and start using data for second track. + +The main additions are: + +- set_metadata +This routine sets the encoder delay and encoder padding. This can be used by +decoder to strip the silence + +- partial drain +This is called when end of file is reached. The userspace can inform DSP that +EOF is reached and now DSP can start skipping padding delay. Also next write +data would belong to next track + Not supported: - Support for VoIP/circuit-switched calls is not the target of thisinclude/sound/compress_driver.h +2
-0@@ -110,6 +110,8 @@ struct snd_compr_ops { struct snd_compr_params *params);int (*get_params)(struct snd_compr_stream *stream,struct snd_codec *params); +int (*set_metadata)(struct snd_compr_stream *stream, +struct snd_compr_metadata *metadata);int (*trigger)(struct snd_compr_stream *stream,int cmd);int (*pointer)(struct snd_compr_stream *stream,struct snd_compr_tstamp *tstamp);include/uapi/sound/compress_offload.h +17
-1@@ -30,7 +30,7 @@#include <sound/compress_params.h> -#define SNDRV_COMPRESS_VERSION SNDRV_PROTOCOL_VERSION(0, 1, 0) +#define SNDRV_COMPRESS_VERSION SNDRV_PROTOCOL_VERSION(0, 1, 1)/** * struct snd_compressed_buffer: compressed buffer * @fragment_size: size of buffer fragment in bytes @@ -122,6 +122,18 @@ struct snd_compr_codec_caps { };/** +* struct snd_compr_metadata: compressed stream metadata +* @encoder_delay: no of samples inserted by the encoder at the beginning +* of the track +* @encoder_padding: no of samples appended by the encoder at the end +* of the track +*/ +struct snd_compr_metadata { +__u32 encoder_delay; +__u32 encoder_padding; +}; + +/** * compress path ioctl definitions * SNDRV_COMPRESS_GET_CAPS: Query capability of DSP * SNDRV_COMPRESS_GET_CODEC_CAPS: Query capability of a codec @@ -145,6 +157,8 @@ struct snd_compr_codec_caps { struct snd_compr_codec_caps)#define SNDRV_COMPRESS_SET_PARAMS _IOW('C', 0x12, struct snd_compr_params)#define SNDRV_COMPRESS_GET_PARAMS _IOR('C', 0x13, struct snd_codec) +#define SNDRV_COMPRESS_SET_METADATA _IOW('C', 0x14,\ + struct snd_compr_metadata)#define SNDRV_COMPRESS_TSTAMP _IOR('C', 0x20, struct snd_compr_tstamp)#define SNDRV_COMPRESS_AVAIL _IOR('C', 0x21, struct snd_compr_avail)#define SNDRV_COMPRESS_PAUSE _IO('C', 0x30)@@ -152,10 +166,12 @@ struct snd_compr_codec_caps {#define SNDRV_COMPRESS_START _IO('C', 0x32)#define SNDRV_COMPRESS_STOP _IO('C', 0x33)#define SNDRV_COMPRESS_DRAIN _IO('C', 0x34) +#define SNDRV_COMPRESS_PARTIAL_DRAIN _IO('C', 0x35)/* * TODO * 1. add mmap support * */ #define SND_COMPR_TRIGGER_DRAIN 7/*FIXME move this to pcm.h */ +#define SND_COMPR_TRIGGER_PARTIAL_DRAIN 8#endif sound/core/compress_offload.c +51
-0@@ -514,6 +514,37 @@ out:return retval; } +static int +snd_compr_set_metadata (struct snd_compr_stream *stream,unsigned long arg) +{ +struct snd_compr_metadata *metadata; +int retval; + +if (!stream->ops->set_metadata) +return -ENXIO ; +/* +* we should allow parameter change only when stream has been +* opened not in other cases +*/ + metadata =kmalloc (sizeof (*metadata),GFP_KERNEL ); +if (!metadata) +return -ENOMEM ; +if (copy_from_user (metadata, (void __user *)arg, +sizeof (*metadata))) { + retval = -EFAULT ; + goto out; + } + +pr_debug ("metadata encoder delay=%x encoder padding=%x\n" , + metadata->encoder_delay, metadata->encoder_padding); + + retval = stream->ops->set_metadata (stream, metadata); + +out: +kfree (metadata); +return retval; +} +static inline int snd_compr_tstamp (struct snd_compr_stream *stream,unsigned long arg) {@@ -594,6 +625,20 @@ static int snd_compr_drain(struct snd_compr_stream *stream)return retval; } +static int snd_compr_partial_drain (struct snd_compr_stream *stream) +{ +int retval; +if (stream->runtime->state ==SNDRV_PCM_STATE_PREPARED || + stream->runtime->state ==SNDRV_PCM_STATE_SETUP ) +return -EPERM ; + retval = stream->ops->trigger (stream,SND_COMPR_TRIGGER_PARTIAL_DRAIN ); +if (retval) +pr_err ("Partial drain returned failure\n" ); +else + stream->runtime->state =SNDRV_PCM_STATE_SETUP ; +return retval; +} +static long snd_compr_ioctl (struct file *f,unsigned int cmd,unsigned long arg) {struct snd_compr_file *data = f->private_data;@@ -623,6 +668,9 @@ static long snd_compr_ioctl(struct file *f, unsigned int cmd, unsigned long arg)case _IOC_NR (SNDRV_COMPRESS_GET_PARAMS ): retval =snd_compr_get_params (stream, arg);break ; +case _IOC_NR (SNDRV_COMPRESS_SET_METADATA ): + retval =snd_compr_set_metadata (stream, arg); +break ;case _IOC_NR (SNDRV_COMPRESS_TSTAMP ): retval =snd_compr_tstamp (stream, arg);break ;@@ -644,6 +692,9 @@ static long snd_compr_ioctl(struct file *f, unsigned int cmd, unsigned long arg)case _IOC_NR (SNDRV_COMPRESS_DRAIN ): retval =snd_compr_drain (stream);break ; +case _IOC_NR (SNDRV_COMPRESS_PARTIAL_DRAIN ): + retval =snd_compr_partial_drain (stream); +break ; }mutex_unlock (&stream->device->lock);return retval;Signature · 1 line
1.7.0.4
Quoted 5 lines · Takashi Iwai
At Tue, 5 Feb 2013 06:21:25 -0800,
Quoted 3 lines · Vinod Koul
From: Jeeja KP <[email protected]>
Quoted 7 lines
Quoted 4 lines
this add new API for sound compress to support gapless playback. As noted in Documentation change, we add API to send metadata of encoder and padding delay to DSP. Also add API for indicating EOF and switching to subsequent track
I can understand that the metadata is somehow handled in DSP for seamless switching,
yes for stripping the encoder delay and encoder padding.
Quoted 3 lines
but the operation with partial drain is a bit unclear.
How is the operation flow? Does it drain until which point?
Right, we want to move from one track to another. And while at this DSP needs to exhaust the buffers from first track and then we should start writing second track. When userspace has finsihed wirting first track it sends partial_drain and DSP decods finsihed off and returns when it is ready to accept second track data. The userland sends the metadata for this followed by data write.
Quoted 2 lines
In your patch, the runtime state is changed to SETUP after the partial drain, so the app is requested to give START again?
Ah, thats a mistake, it should be in running state, no START again :)
--
~Vinod
At Wed, 6 Feb 2013 06:09:32 -0800,
Quoted 15 lines · Vinod Koul
Quoted 5 lines · Takashi Iwai
At Tue, 5 Feb 2013 06:21:25 -0800,
Quoted 3 lines · Vinod Koul
From: Jeeja KP <[email protected]>
Quoted 7 lines
Quoted 4 lines
this add new API for sound compress to support gapless playback. As noted in Documentation change, we add API to send metadata of encoder and padding delay to DSP. Also add API for indicating EOF and switching to subsequent track
I can understand that the metadata is somehow handled in DSP for seamless switching,
yes for stripping the encoder delay and encoder padding.
Quoted 8 lines
Quoted 3 lines
but the operation with partial drain is a bit unclear.
How is the operation flow? Does it drain until which point?Right, we want to move from one track to another. And while at this DSP needs to exhaust the buffers from first track and then we should start writing second track. When userspace has finsihed wirting first track it sends
partial_drainand DSP decods finsihed off and returns when it is ready to accept second track data. The userland sends the metadata for this followed by data write.
Hm, could you depict the flow? Will be like below?
- Open
- Get caps / codec caps
- Set params
- Set metadata of the first track
- Fill data of the first track
- Trigger START
- User-space finished sending all, then trigger PARTIAL_DRAIN
- Set metadata of the next track
- Fill data of the next track
....
Quoted 3 lines
Quoted 2 lines
In your patch, the runtime state is changed to SETUP after the partial drain, so the app is requested to give START again?
Ah, thats a mistake, it should be in running state, no START again :)
OK, then PARTIAL_DRAIN doesn't sound intuitive.
Something like CONTINUE_TO_NEXT?
(I'm not particularly good at naming, so someone must have a better
idea...)
Takashi
Quoted 9 lines · Takashi Iwai
Quoted 6 lines
Quoted 1 line
How is the operation flow? Does it drain until which point?
Right, we want to move from one track to another. And while at this DSP needs to exhaust the buffers from first track and then we should start writing second track. When userspace has finsihed wirting first track it sends
partial_drainand DSP decods finsihed off and returns when it is ready to accept second track data. The userland sends the metadata for this followed by data write.Hm, could you depict the flow? Will be like below?
Quoted 7 lines
- Open
- Get caps / codec caps
- Set params
- Set metadata of the first track
- Fill data of the first track
- Trigger START
- User-space finished sending all, then triggerPARTIAL_DRAIN
PARTAIL_DRAIN is returned when DSP has finished most of data from first track and is ready to accept second track data
Quoted 2 lines
- Set metadata of the next track
- Fill data of the next track
DSP switches to second track
Keep writing till we finish track, then trigger PARTIAL_DRAIN Continue till you reach end of album
Then trigger DRAIN
Close
I will add this to documentation patch as well...
Quoted 2 lines
OK, then
PARTIAL_DRAINdoesn't sound intuitive.
Something likeCONTINUE_TO_NEXT?
Sure, anyone else with a better name :-)
Quoted 3 lines
(I'm not particularly good at naming, so someone must have a better
idea...)
Welcome to club, I wont mind naming after you :D
--
~Vinod
A message that is not in the archive (sent off-list or not yet received): 37A133201056E44A80888420ECA881BF1093DC5F68@EXCMB2.wolfsonmicro.main
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 28 lines · Richard Fitzgerald
Documentation/sound/alsa/compress_offload.txt Modifications include: · line 168
Quoted 4 lines
+- partial drain
Documentation/sound/alsa/compress_offload.txt +3
-0+This is called when end of file is reached. The userspace can inform DSP that +EOF is reached and now DSP can start skipping padding delay. Also next write +data would belong to next trackWe're really doing two different tasks inside the "partial drain". The name, and the reason we are doing two tasks in one, comes from a particular higher- layer scenario, but there's no reason the driver API need use the same terminology as one particular application of the functionality. The two tasks are:
1) Tell the DSP that we have sent all data for the current track and following data will be for the next track. This lets the DSP lay down a marker for where it should strip padding at the end of a track, and know it should be expecting more data for another track to follow gaplessly where it must strip the encoder delay from the start.
2) Ask for notification when DSP has reached the changeover point between the playback of the two tracks
I think it would be more logical and less confusing not to combine the two into a single ioctl. Instead, add only one new ioctl specifically to provide the track changeover hint, something like
SNDRV_COMPRESS_NEXT_TRACK(meaning DSP should expect data for next track to follow). Don't add a new drain, just use the existingSNDRV_COMPRESS_DRAIN- the driver/DSP can make a decision whether it needs to do something special with the drain if we have told it that we will be sending it some more data for a following track.
Quoted 4 lines
So the
SNDRV_COMPRESS_PARTIAL_DRAINin this patch would become1) Send
SNDRV_COMPRESS_NEXT_TRACK
2) SendSNDRV_COMPRESS_DRAIN
The problem would be in that case the defination of SNDRV_COMPRESS_DRAIN which expects the decoder to completely drain its buffers and come to complete halt. This would also mean the framework will treat a drained stream as stopped and needs a new start. Certainly we dont want that in this case. So we can't use SNDRV_COMPRESS_DRAIN to indicate. Yes we can put conditional check but IMO that would overtly complicate this. If we are not doing proper drian lets not call it that.
But I think I like the idea of splitting the two up to do a cleaner interface. Let me check this...
Quoted 7 lines
When we reach the final track we just do
SNDRV_COMPRESS_DRAINto wait for it to finish.If we setup a next track by doing
SNDRV_COMPRESS_NEXT_TRACKand then change our mind and decide that this is going to be the last track, we do aSNDRV_COMPRESS_DRAINwithout sending any next track data, then we doSNDRV_COMPRESS_STOP
Nope that is where we would have issue. You dont call SNDRV_COMPRESS_STOP for compressed streams. You have to wait till DSP has decoded and rendered data for the last track which can be done by using SNDRV_COMPRESS_DRAIN only. SNDRV_COMPRESS_STOP should not be called in this case.
--
~Vinod
At Wed, 6 Feb 2013 18:18:07 -0800,
Quoted 42 lines · Vinod Koul
Quoted 28 lines · Richard Fitzgerald
Documentation/sound/alsa/compress_offload.txt Modifications include: · line 168
Quoted 4 lines
+- partial drain
Documentation/sound/alsa/compress_offload.txt +3
-0+This is called when end of file is reached. The userspace can inform DSP that +EOF is reached and now DSP can start skipping padding delay. Also next write +data would belong to next trackWe're really doing two different tasks inside the "partial drain". The name, and the reason we are doing two tasks in one, comes from a particular higher- layer scenario, but there's no reason the driver API need use the same terminology as one particular application of the functionality. The two tasks are:
1) Tell the DSP that we have sent all data for the current track and following data will be for the next track. This lets the DSP lay down a marker for where it should strip padding at the end of a track, and know it should be expecting more data for another track to follow gaplessly where it must strip the encoder delay from the start.
2) Ask for notification when DSP has reached the changeover point between the playback of the two tracks
I think it would be more logical and less confusing not to combine the two into a single ioctl. Instead, add only one new ioctl specifically to provide the track changeover hint, something like
SNDRV_COMPRESS_NEXT_TRACK(meaning DSP should expect data for next track to follow). Don't add a new drain, just use the existingSNDRV_COMPRESS_DRAIN- the driver/DSP can make a decision whether it needs to do something special with the drain if we have told it that we will be sending it some more data for a following track.Quoted 4 lines
So the
SNDRV_COMPRESS_PARTIAL_DRAINin this patch would become1) Send
SNDRV_COMPRESS_NEXT_TRACK
2) SendSNDRV_COMPRESS_DRAINThe problem would be in that case the defination of
SNDRV_COMPRESS_DRAINwhich expects the decoder to completely drain its buffers and come to complete halt. This would also mean the framework will treat a drained stream as stopped and needs a new start. Certainly we dont want that in this case. So we can't useSNDRV_COMPRESS_DRAINto indicate. Yes we can put conditional check but IMO that would overtly complicate this. If we are not doing proper drian lets not call it that.
Quoted 2 lines
But I think I like the idea of splitting the two up to do a cleaner interface. Let me check this...
In the way above, NEXT_TRACK is no longer a trigger op, but it's rather an operational option. This would set a flag something like
compr->drain_mode = SND_COMPR_DRAIN_RESTART;
while its default is SND_COMPR_DRAIN_STOP.
(Or, set compr->next_track_after_drain = true, or whatever.)
Though, I'm also not sure whether the scenario above improves the situation better...
Takashi
A message that is not in the archive (sent off-list or not yet received): [email protected]
At Thu, 7 Feb 2013 11:37:38 +0000,
Quoted 29 lines · Richard Fitzgerald
Quoted 10 lines
Quoted 3 lines
1) Send
SNDRV_COMPRESS_NEXT_TRACK
2) SendSNDRV_COMPRESS_DRAINThe problem would be in that case the defination of
SNDRV_COMPRESS_DRAINwhich expects the decoder to completely drain its buffers and come to complete halt. This would also mean the framework will treat a drained stream as stopped and needs a new start. Certainly we dont want that in this case. So we can't useSNDRV_COMPRESS_DRAINto indicate. Yes we can put conditional check but IMO that would overtly complicate this. If we are not doing proper drian lets not call it that.Ok, so let's keep the partial drain but split out the
NEXT_TRACKhint so we keep the two operations separate. It's clearer to understand.NEXT_TRACKtells DSP to prepare for data for new track,PARTIAL_DRAINasks DSP when it has played all data of current track. I don't think that application _must_ drain when doing gapless, only if it cares to know when DSP reaches the join between tracks. Possible problem - application callsNEXT_TRACKandPARTIAL_DRAINwhen DSP has already reached end of current track, so what doesPARTIAL_DRAINmean in this case? Which track is it draining? Maybe define that it always refers to the track _before_ the most recentNEXT_TRACKcall?Also can we make metadata a key-pair list as Takashi suggested, with unlimited length. Something like
struct snd_compr_metadata_pair { enum snd_compr_metadata_key key; u32 value; };
Quoted 7 lines
struct snd_compr_metadata { int count; /* number of actual entries in following array */ struct snd_compr_metadata_pair *pairs; };Should we union the u32 value with a u8* buf to allow for any case where the value needs to be a buffer of binary data?
Oh no, don't do it. Never.
In general,
- Never put types that depending on the architecture in struct passed to ioctls (e.g. long, pointer)
- Never use bit fields for ABI
- If 64bit value is used, think of alignment; safer to add packed attribute
So, in general, use only u32, s16, or whatever, the types with explicit size.
Takashi
Quoted 10 lines · Richard Fitzgerald
Quoted 9 lines
Quoted 2 lines
1) Send
SNDRV_COMPRESS_NEXT_TRACK
2) SendSNDRV_COMPRESS_DRAINThe problem would be in that case the defination of
SNDRV_COMPRESS_DRAINwhich expects the decoder to completely drain its buffers and come to complete halt. This would also mean the framework will treat a drained stream as stopped and needs a new start. Certainly we dont want that in this case. So we can't useSNDRV_COMPRESS_DRAINto indicate. Yes we can put conditional check but IMO that would overtly complicate this. If we are not doing proper drian lets not call it that.
Quoted 5 lines
Ok, so let's keep the partial drain but split out the
NEXT_TRACKhint so we keep the two operations separate. It's clearer to understand.NEXT_TRACKtells DSP to prepare for data for new track,PARTIAL_DRAINasks DSP when it has played all data of current track. I don't think that application _must_ drain when doing gapless, only if it cares to know when DSP reaches the join between tracks.
If we split this then I dont see why next write/meta_data call should wait. Then the PARTIAL_DRAIN should be used for knowing in userpsace that switch happened, right?
Quoted 3 lines
Possible problem - application calls
NEXT_TRACKandPARTIAL_DRAINwhen DSP has already reached end of current track, so what doesPARTIAL_DRAINmean in this case?
It should just return immediately, and if your output is starved you are in error state, that can happen when you dont give data as well.
Quoted 7 lines
Also can we make metadata a key-pair list as Takashi suggested, with unlimited length. Something like
struct snd_compr_metadata_pair { enum snd_compr_metadata_key key; u32 value; };
No enums... Thats not how kernel ABI is designed.
I am leaning towards key/value pair, but is this fair assumption that value is always 32bit, what if we have larger single value, then should we split?
Quoted 8 lines
struct snd_compr_metadata { int count; /* number of actual entries in following array */ struct snd_compr_metadata_pair *pairs; };Should we union the u32 value with a u8* buf to allow for any case where the value needs to be a buffer of binary data? Is there any known case like this?
Again nope, that aint good from ABI POV.
What I think it should be is:
enum {
SNDRV_COMPRESS_ENCODER_PADDING = 1,
SNDRV_COMPRESS_ENCODER_DELAY = 2,
};
struct snd_compr_metadata {
u32 key;
u32 value;
};--
~Vinod
Quoted 44 lines · Takashi Iwai
At Wed, 6 Feb 2013 18:18:07 -0800,
Quoted 42 lines · Vinod Koul
Quoted 28 lines · Richard Fitzgerald
Documentation/sound/alsa/compress_offload.txt Modifications include: · line 168
Quoted 4 lines
+- partial drain
Documentation/sound/alsa/compress_offload.txt +3
-0+This is called when end of file is reached. The userspace can inform DSP that +EOF is reached and now DSP can start skipping padding delay. Also next write +data would belong to next trackWe're really doing two different tasks inside the "partial drain". The name, and the reason we are doing two tasks in one, comes from a particular higher- layer scenario, but there's no reason the driver API need use the same terminology as one particular application of the functionality. The two tasks are:
1) Tell the DSP that we have sent all data for the current track and following data will be for the next track. This lets the DSP lay down a marker for where it should strip padding at the end of a track, and know it should be expecting more data for another track to follow gaplessly where it must strip the encoder delay from the start.
2) Ask for notification when DSP has reached the changeover point between the playback of the two tracks
I think it would be more logical and less confusing not to combine the two into a single ioctl. Instead, add only one new ioctl specifically to provide the track changeover hint, something like
SNDRV_COMPRESS_NEXT_TRACK(meaning DSP should expect data for next track to follow). Don't add a new drain, just use the existingSNDRV_COMPRESS_DRAIN- the driver/DSP can make a decision whether it needs to do something special with the drain if we have told it that we will be sending it some more data for a following track.Quoted 4 lines
So the
SNDRV_COMPRESS_PARTIAL_DRAINin this patch would become1) Send
SNDRV_COMPRESS_NEXT_TRACK
2) SendSNDRV_COMPRESS_DRAINThe problem would be in that case the defination of
SNDRV_COMPRESS_DRAINwhich expects the decoder to completely drain its buffers and come to complete halt. This would also mean the framework will treat a drained stream as stopped and needs a new start. Certainly we dont want that in this case. So we can't useSNDRV_COMPRESS_DRAINto indicate. Yes we can put conditional check but IMO that would overtly complicate this. If we are not doing proper drian lets not call it that.
Quoted 8 lines
Quoted 2 lines
But I think I like the idea of splitting the two up to do a cleaner interface. Let me check this...
In the way above,
NEXT_TRACKis no longer a trigger op, but it's rather an operational option. This would set a flag something likecompr->drain_mode = SND_COMPR_DRAIN_RESTART;while its default is
SND_COMPR_DRAIN_STOP.
(Or, setcompr->next_track_after_drain= true, or whatever.)
So the basic difference would be that stream is never stopped in gapless case. The stream should retain running state maybe trasition to intermediate one for book keeping but then needs to go back to running.
But, should framework really care about that as DSP is doing handling!
--
~Vinod