Skip to contentSkip to conversation

problem w/ arecord "split interleaved files" feature

PATCH In review Dave Maley 2 participants 6 messages Apr 25, 2006 – Apr 30, 2006

lore
Share & tools
lore

Dave Maley

author
#0 lore raw

PATCH 1/1 problem w/ arecord "split interleaved files" feature

Hello,

My first post to the alsa-devel list, and I'm not subscribed so please
cc: me directly on any responses ... This is in reference to a patch which was incorporated into alsa-utils-1.0.11, originally posted here:

http://thread.gmane.org/gmane.linux.alsa.devel/29025/focus=29098

The max_filesize for wave files as currently implemented it's actually splitting files a good bit before reaching 2GB.

A little background as to why this matters (to me at least). I use arecord to transfer DAT's to my computer, and many of these DAT's are 3 hours in length. So for this I'd typically do something to the effect of:

[dave@flanders ~]$ arecord -t wav -f dat -d 10800 foo.wav

Since max_filesize is set to 2000000000 arecord splits the resulting file a bit over 170 minutes. Not a huge problem, except I pipe the above command to sox for down-sampling to 44.1KHz:

[dave@flanders ~]$ arecord -q -t wav -f dat -d 10800 | sox -t wav -r 48000 -c 2 - -t wav -r 44100 -c 2 foo.wav resample

It seems this new split feature doesn't work properly when writing to a pipe (or at least when piping to sox). A quick look at an strace shows constant write errors, and foo.wav grows in size until the disk is full.

A quick fix for this is to set max_filesize to 2147483648. Not sure if this is a "proper" fix however I have tested it and it seems to work just fine for me. Please consider the attached patch.

Thanks,
Dave

aplay/aplay.c renamed from aplay/aplay.c.orig +1 -1
@@ -128,7 +128,7 @@
 } fmt_rec_table[] = {
 	{	NULL,		end_raw,	N_("raw data"),		LLONG_MAX },
 	{	begin_voc,	end_voc,	N_("VOC"),		16000000 },
-	{	begin_wave,	end_wave,	N_("WAVE"),		2000000000 },
+	{	begin_wave,	end_wave,	N_("WAVE"),		2147483648 },
 	{	begin_au,	end_au,		N_("Sparc Audio"),	LLONG_MAX }
 };
 
#1 lore raw

At Tue, 25 Apr 2006 01:43:24 -0400,

Quoted 26 lines · Dave Maley

Hello,

My first post to the alsa-devel list, and I'm not subscribed so please
cc: me directly on any responses ... This is in reference to a patch which was incorporated into alsa-utils-1.0.11, originally posted here:

http://thread.gmane.org/gmane.linux.alsa.devel/29025/focus=29098

The max_filesize for wave files as currently implemented it's actually splitting files a good bit before reaching 2GB.

A little background as to why this matters (to me at least). I use arecord to transfer DAT's to my computer, and many of these DAT's are 3 hours in length. So for this I'd typically do something to the effect of:

[dave@flanders ~]$ arecord -t wav -f dat -d 10800 foo.wav

Since max_filesize is set to 2000000000 arecord splits the resulting file a bit over 170 minutes. Not a huge problem, except I pipe the above command to sox for down-sampling to 44.1KHz:

Quoted 7 lines

[dave@flanders ~]$ arecord -q -t wav -f dat -d 10800 | sox -t wav -r 48000 -c 2 - -t wav -r 44100 -c 2 foo.wav resample

It seems this new split feature doesn't work properly when writing to a pipe (or at least when piping to sox). A quick look at an strace shows constant write errors, and foo.wav grows in size until the disk is full.

I guess it's rather a bug of sox. arecord quits when the data size reaches to max_filesize in the case of pipe output (since obviously you cannot split the output). sox should quits, too, when receiving EPIPE error.

Quoted 3 lines

A quick fix for this is to set max_filesize to 2147483648. Not sure if this is a "proper" fix however I have tested it and it seems to work just fine for me. Please consider the attached patch.

Well, I'm not sure whether exactly 2GB is allowed.

Does anyone know the exact specification of WAV (PCM) format?

Takashi

-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

author
#2 lore raw

Thanks for the quick response ...

Quoted 4 lines
Quoted 4 lines

[dave@flanders ~]$ arecord -q -t wav -f dat -d 10800 | sox -t wav -r 48000 -c 2 - -t wav -r 44100 -c 2 foo.wav resample

Quoted 8 lines
Quoted 3 lines

It seems this new split feature doesn't work properly when writing to a pipe (or at least when piping to sox). A quick look at an strace shows constant write errors, and foo.wav grows in size until the disk is full.

I guess it's rather a bug of sox. arecord quits when the data size reaches to max_filesize in the case of pipe output (since obviously you cannot split the output). sox should quits, too, when receiving EPIPE error.

Hmm, this doesn't seem to match the behavior I'm seeing. It's very possible I'm misunderstanding something here, but this is what I see:

Using the above arecord | sox command when the foo.wav file reaches 2000000000 in size and the split occurs it'll actually be split into foo-01.wav and foo-02.wav, and I believe that this rename of foo.wav to foo-01.wav is where the problem may be occurring. I can capture strace output at the time of the split, if it's at all helpful.

Quoted 5 lines
Quoted 3 lines

A quick fix for this is to set max_filesize to 2147483648. Not sure if this is a "proper" fix however I have tested it and it seems to work just fine for me. Please consider the attached patch.

Well, I'm not sure whether exactly 2GB is allowed.

If not then I'd be happy with just increasing this max_filesize to 2073600044 as that's what my 180 minute 48KHz files end up being ... 8)

Thanks again for your feedback on this.

~Dave

-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

#3 lore raw

At Tue, 25 Apr 2006 17:07:35 -0400,

Quoted 16 lines · Dave Maley

Thanks for the quick response ...

Quoted 4 lines
Quoted 4 lines

[dave@flanders ~]$ arecord -q -t wav -f dat -d 10800 | sox -t wav -r 48000 -c 2 - -t wav -r 44100 -c 2 foo.wav resample

Quoted 8 lines
Quoted 3 lines

It seems this new split feature doesn't work properly when writing to a pipe (or at least when piping to sox). A quick look at an strace shows constant write errors, and foo.wav grows in size until the disk is full.

I guess it's rather a bug of sox. arecord quits when the data size reaches to max_filesize in the case of pipe output (since obviously you cannot split the output). sox should quits, too, when receiving EPIPE error.

Quoted 8 lines

Hmm, this doesn't seem to match the behavior I'm seeing. It's very possible I'm misunderstanding something here, but this is what I see:

Using the above arecord | sox command when the foo.wav file reaches 2000000000 in size and the split occurs it'll actually be split into foo-01.wav and foo-02.wav, and I believe that this rename of foo.wav to foo-01.wav is where the problem may be occurring. I can capture strace output at the time of the split, if it's at all helpful.

I found a stupid bug there. arecord doesn't quit properly at max_filesize due to a wrong comparison. The fix patch is below.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

A quick fix for this is to set max_filesize to 2147483648. Not sure if this is a "proper" fix however I have tested it and it seems to work just fine for me. Please consider the attached patch.

Well, I'm not sure whether exactly 2GB is allowed.

If not then I'd be happy with just increasing this max_filesize to 2073600044 as that's what my 180 minute 48KHz files end up being ... 8)

Maybe 2GB is OK. I'll fix it too.

Takashi

aplay/aplay.c +4 -2
diff -r 2dacb18877f3 aplay/aplay.c
@@ -2165,13 +2165,15 @@ static void capture(char *name_)
 			   2) all requested samples/bytes have been captured (cur>0)
 			   3) filesize threshold was reached (fdcount<wrthreshold)
 			*/
-		} while (file_type == FORMAT_RAW && !timelimit && cur>0 && fdcount<fmt_rec_table[file_type].max_filesize);
+		} while (file_type == FORMAT_RAW && !timelimit && cur>0 &&
+			 fdcount<fmt_rec_table[file_type].max_filesize);
 
 		/* finish sample container */
 		fmt_rec_table[file_type].end(fd);
 
 		/* repeat the loop when format is raw without timelimit or filesize threshold was reached */
-	} while((file_type == FORMAT_RAW && !timelimit) || fdcount>=fmt_rec_table[file_type].max_filesize);
+	} while((file_type == FORMAT_RAW && !timelimit) ||
+		fdcount < fmt_rec_table[file_type].max_filesize);
 }
 
 void playbackv_go(int* fds, unsigned int channels, size_t loaded, off64_t count, int rtype, char **names)

-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

author
#4 lore raw
Quoted 2 lines

I found a stupid bug there. arecord doesn't quit properly at max_filesize due to a wrong comparison. The fix patch is below.

Excellent - Thanks much. I'll test this tonight and report back the results.

Quoted 1 line

Maybe 2GB is OK. I'll fix it too.

No idea what the spec says but I've recorded test wave files which were exactly 2GB and they didn't seem to have any problems (played fine in totem and opened properly in a wave editor).

Thanks again for you prompt responses and patch!

~Dave

-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

author
#5 lore raw
In reply to Takashi Iwai

Hi Takashi,

Quoted 2 lines

I found a stupid bug there. arecord doesn't quit properly at max_filesize due to a wrong comparison. The fix patch is below.

I finally had a chance to test this patch and it indeed resolves the problem I was running into.

Quoted 1 line

Maybe 2GB is OK. I'll fix it too.

Excellent. Thanks much for looking into this.

~Dave

-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642