Skip to contentSkip to conversation

Digital Audio Extraction with ATAPI drives far from perfect

DISCUSSION Sebastian 11 participants 33 messages Jan 3, 2006 – Jan 10, 2006 4 messages not in the archive Uniform cdrom driver

lore
Share & tools
lore

Sebastian

author
#0 lore raw

Hello all!

I'd be kind if you would cc me in case you reply as I'm not (yet) subscribed to this list.

I'm a music buff :) Well, I guess most people are in one way or the other. But in addition to that I like bitperfect ripping very much. Since some time this doesn't work well with linux anymore.

I'd like to know if there's something in the making already or what could be done to solve this issue. A supporting layer for the ide system, a new driver maybe? I think this can't be fixed in userspace, right?

I ripped an audio disc in different ways and compared the results using md5sum. As you can see the wav data is perfect when using ide-scsi emulation. On the contrary, using ide lead to errors.

Linux version 2.6.15 (root@section_eight) (gcc-Version 3.4.4 (Gentoo 3.4.4-r1, ssp-3.4.4-1.0, pie-8.7.8))

In all cases I used the same drive, a NEC ND-4550A dvd writer.

The first series of wav files was ripped with EAC in Windows and is bitperfect:

8cab5ca4820a753ebb3cb7e3c5c34e6a  01.Man In A Suitcase.wav
187f90f900cffd36ce56e97a3bcf595e  02.Box Of Six.wav
8cc168bb50e80a06693a01a9cd34bbdf  03.Mysterons.wav
e82169e5ea1b441b80db96fce12fd109  04.Justified.wav
8d807b7ac19f90049aec6ff177e9b486  05.Department S.wav
130306e9a564c844d5269256f38afca7  06.Area Code 51.wav
96489dbfcab8f97e7e450cf8db2c8aaa  07.Has To Be.wav
4597a6ed75e201916a3479f05eb86405  08.No. 5.wav
f978327a98fc6359be2fc25eb865211d  09.Among The Cybermen.wav
e316e140b4b0cd66e5822edae22dadb1  10.Unspeakable Elvis.wav
3792a680b1ba729de9185043d331186f  11.Xodiak.wav
ba534fd8eb42dd84aa7b59ab3ae6f132  12.Northern Wisdom.wav
d6346ab76696dddf735a5b752aa7888b  13.Trinity Road.wav

The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.

The third series was done with ide-cd. Erroneous data is marked with a (!):

e8319ccc20d053557578b9ca3eb368dd  track01.cdda.wav (!)
cb978f86ddc18c9df1b7e91705380bc5  track02.cdda.wav (!)
35f1b296d72a8708d03aeb540a3b4f30  track03.cdda.wav (!)
e82169e5ea1b441b80db96fce12fd109  track04.cdda.wav
8d807b7ac19f90049aec6ff177e9b486  track05.cdda.wav
02561939763d67aacf23157c09966a89  track06.cdda.wav (!)
9724b0a3e2295084613da9df7397ae6d  track07.cdda.wav (!)
c2d85b3d10428aad66664d0fb3e4c71a  track08.cdda.wav (!)
5116b2fae44b8b86fbf40b9bac9a8268  track09.cdda.wav (!)
9e6a5ab2dab76e1677667f586895293a  track10.cdda.wav (!)
3792a680b1ba729de9185043d331186f  track11.cdda.wav
ba534fd8eb42dd84aa7b59ab3ae6f132  track12.cdda.wav
d6346ab76696dddf735a5b752aa7888b  track13.cdda.wav

Sebastian

Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
#1 lore raw
Quoted 45 lines · Sebastian

Hello all!

I'd be kind if you would cc me in case you reply as I'm not (yet) subscribed to this list.

I'm a music buff :) Well, I guess most people are in one way or the other. But in addition to that I like bitperfect ripping very much. Since some time this doesn't work well with linux anymore.

I'd like to know if there's something in the making already or what could be done to solve this issue. A supporting layer for the ide system, a new driver maybe? I think this can't be fixed in userspace, right?

I ripped an audio disc in different ways and compared the results using md5sum. As you can see the wav data is perfect when using ide-scsi emulation. On the contrary, using ide lead to errors.

Linux version 2.6.15 (root@section_eight) (gcc-Version 3.4.4 (Gentoo 3.4.4-r1, ssp-3.4.4-1.0, pie-8.7.8))

In all cases I used the same drive, a NEC ND-4550A dvd writer.

The first series of wav files was ripped with EAC in Windows and is bitperfect:

8cab5ca4820a753ebb3cb7e3c5c34e6a  01.Man In A Suitcase.wav
187f90f900cffd36ce56e97a3bcf595e  02.Box Of Six.wav
8cc168bb50e80a06693a01a9cd34bbdf  03.Mysterons.wav
e82169e5ea1b441b80db96fce12fd109  04.Justified.wav
8d807b7ac19f90049aec6ff177e9b486  05.Department S.wav
130306e9a564c844d5269256f38afca7  06.Area Code 51.wav
96489dbfcab8f97e7e450cf8db2c8aaa  07.Has To Be.wav
4597a6ed75e201916a3479f05eb86405  08.No. 5.wav
f978327a98fc6359be2fc25eb865211d  09.Among The Cybermen.wav
e316e140b4b0cd66e5822edae22dadb1  10.Unspeakable Elvis.wav
3792a680b1ba729de9185043d331186f  11.Xodiak.wav
ba534fd8eb42dd84aa7b59ab3ae6f132  12.Northern Wisdom.wav
d6346ab76696dddf735a5b752aa7888b  13.Trinity Road.wav

The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.

The third series was done with ide-cd. Erroneous data is marked with a (!):

Quoted 13 lines
e8319ccc20d053557578b9ca3eb368dd  track01.cdda.wav (!)
cb978f86ddc18c9df1b7e91705380bc5  track02.cdda.wav (!)
35f1b296d72a8708d03aeb540a3b4f30  track03.cdda.wav (!)
e82169e5ea1b441b80db96fce12fd109  track04.cdda.wav
8d807b7ac19f90049aec6ff177e9b486  track05.cdda.wav
02561939763d67aacf23157c09966a89  track06.cdda.wav (!)
9724b0a3e2295084613da9df7397ae6d  track07.cdda.wav (!)
c2d85b3d10428aad66664d0fb3e4c71a  track08.cdda.wav (!)
5116b2fae44b8b86fbf40b9bac9a8268  track09.cdda.wav (!)
9e6a5ab2dab76e1677667f586895293a  track10.cdda.wav (!)
3792a680b1ba729de9185043d331186f  track11.cdda.wav
ba534fd8eb42dd84aa7b59ab3ae6f132  track12.cdda.wav
d6346ab76696dddf735a5b752aa7888b  track13.cdda.wav

Can you try and see how, say, track01 differ? Is it single bytes, chunks of 2352 bytes, or?

Signature · 1 line
Jens Axboe
#2 lore raw
Quoted 60 lines · Jens Axboe
Quoted 45 lines · Sebastian

Hello all!

I'd be kind if you would cc me in case you reply as I'm not (yet) subscribed to this list.

I'm a music buff :) Well, I guess most people are in one way or the other. But in addition to that I like bitperfect ripping very much. Since some time this doesn't work well with linux anymore.

I'd like to know if there's something in the making already or what could be done to solve this issue. A supporting layer for the ide system, a new driver maybe? I think this can't be fixed in userspace, right?

I ripped an audio disc in different ways and compared the results using md5sum. As you can see the wav data is perfect when using ide-scsi emulation. On the contrary, using ide lead to errors.

Linux version 2.6.15 (root@section_eight) (gcc-Version 3.4.4 (Gentoo 3.4.4-r1, ssp-3.4.4-1.0, pie-8.7.8))

In all cases I used the same drive, a NEC ND-4550A dvd writer.

The first series of wav files was ripped with EAC in Windows and is bitperfect:

8cab5ca4820a753ebb3cb7e3c5c34e6a  01.Man In A Suitcase.wav
187f90f900cffd36ce56e97a3bcf595e  02.Box Of Six.wav
8cc168bb50e80a06693a01a9cd34bbdf  03.Mysterons.wav
e82169e5ea1b441b80db96fce12fd109  04.Justified.wav
8d807b7ac19f90049aec6ff177e9b486  05.Department S.wav
130306e9a564c844d5269256f38afca7  06.Area Code 51.wav
96489dbfcab8f97e7e450cf8db2c8aaa  07.Has To Be.wav
4597a6ed75e201916a3479f05eb86405  08.No. 5.wav
f978327a98fc6359be2fc25eb865211d  09.Among The Cybermen.wav
e316e140b4b0cd66e5822edae22dadb1  10.Unspeakable Elvis.wav
3792a680b1ba729de9185043d331186f  11.Xodiak.wav
ba534fd8eb42dd84aa7b59ab3ae6f132  12.Northern Wisdom.wav
d6346ab76696dddf735a5b752aa7888b  13.Trinity Road.wav

The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.

The third series was done with ide-cd. Erroneous data is marked with a (!):

Quoted 13 lines
e8319ccc20d053557578b9ca3eb368dd  track01.cdda.wav (!)
cb978f86ddc18c9df1b7e91705380bc5  track02.cdda.wav (!)
35f1b296d72a8708d03aeb540a3b4f30  track03.cdda.wav (!)
e82169e5ea1b441b80db96fce12fd109  track04.cdda.wav
8d807b7ac19f90049aec6ff177e9b486  track05.cdda.wav
02561939763d67aacf23157c09966a89  track06.cdda.wav (!)
9724b0a3e2295084613da9df7397ae6d  track07.cdda.wav (!)
c2d85b3d10428aad66664d0fb3e4c71a  track08.cdda.wav (!)
5116b2fae44b8b86fbf40b9bac9a8268  track09.cdda.wav (!)
9e6a5ab2dab76e1677667f586895293a  track10.cdda.wav (!)
3792a680b1ba729de9185043d331186f  track11.cdda.wav
ba534fd8eb42dd84aa7b59ab3ae6f132  track12.cdda.wav
d6346ab76696dddf735a5b752aa7888b  track13.cdda.wav
Quoted 2 lines

Can you try and see how, say, track01 differ? Is it single bytes, chunks of 2352 bytes, or?

Oh, and try and disable DMA on the cd driver and repeat your results with ide-cd. It uses DMA, where ide-scsi does not. Dunno what Windows does. It could just be a problem with your drive and DMA enabled rips.

Signature · 1 line
Jens Axboe
#3 lore raw

Jens Axboe wrote:
...

Quoted 2 lines

Oh, and try and disable DMA on the cd driver and repeat your results with ide-cd. It uses DMA, where ide-scsi does not.

Eh? Sure it does!

#4 lore raw
Quoted 6 lines · Mark Lord

Jens Axboe wrote:
...

Quoted 2 lines

Oh, and try and disable DMA on the cd driver and repeat your results with ide-cd. It uses DMA, where ide-scsi does not.

Eh? Sure it does!

Only for transfers multiple of 1k, which the DMA ripping is not.

Signature · 1 line
Jens Axboe
author
#5 lore raw
In reply to Jens Axboe

Hello all! Hi Jens!

I'd be kind if you would cc me in case you reply as I'm not (yet) subscribed to this list.

Quoted 8 lines · Jens Axboe
Quoted 3 lines · Jens Axboe

Can you try and see how, say, track01 differ? Is it single bytes, chunks of 2352 bytes, or?

Oh, and try and disable DMA on the cd driver and repeat your results with ide-cd. It uses DMA, where ide-scsi does not. Dunno what Windows does. It could just be a problem with your drive and DMA enabled rips.

Hi Jens,

I did as you said and disabled dma:

/dev/hdc:

 IO_support   =  1 (32-bit)
 unmaskirq    =  1 (on)
 using_dma    =  0 (off)
 keepsettings =  0 (off)
 readonly     =  0 (off)

readahead = 256 (on)
HDIO_GETGEO failed: Invalid argument

Then I reripped the whole disc. Now this is remarkable. The rip still has errors, but the errors are not only in the same tracks. They are exactly the same errors (md5sums are equal to the ones I yielded from using ide-cd before):

e8319ccc20d053557578b9ca3eb368dd  track01.cdda.wav (!)
cb978f86ddc18c9df1b7e91705380bc5  track02.cdda.wav (!)
35f1b296d72a8708d03aeb540a3b4f30  track03.cdda.wav (!)
e82169e5ea1b441b80db96fce12fd109  track04.cdda.wav
8d807b7ac19f90049aec6ff177e9b486  track05.cdda.wav
02561939763d67aacf23157c09966a89  track06.cdda.wav (!)
9724b0a3e2295084613da9df7397ae6d  track07.cdda.wav (!)
c2d85b3d10428aad66664d0fb3e4c71a  track08.cdda.wav (!)
5116b2fae44b8b86fbf40b9bac9a8268  track09.cdda.wav (!)
9e6a5ab2dab76e1677667f586895293a  track10.cdda.wav (!)
3792a680b1ba729de9185043d331186f  track11.cdda.wav
ba534fd8eb42dd84aa7b59ab3ae6f132  track12.cdda.wav
d6346ab76696dddf735a5b752aa7888b  track13.cdda.wav

I used the wav compare function in EAC.

1. wav ripped by EAC
####################

What happened? Where?
-------------------------------------------------

Different samples       0:04:08.318 - 0:04:08.362
2100 missing samples    0:04:08.359
Different samples       0:04:08.430 - 0:04:08.433
Different samples       0:04:09.348 - 0:04:09.398

2. wav ripped by cdparanoia
###########################

What happened? Where?
-------------------------------------------------

Different samples       0:04:08.318 - 0:04:08.362
2039 missing samples    0:04:08.414
Different samples       0:04:08.431 - 0:04:08.434
Different samples       0:04:09.349 - 0:04:09.399

I'm sorry if this isn't what you had in mind when you told me to compare the wav files. If it doesn't help what can I do to compare the files to your liking?

Sebastian

Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
#6 lore raw
Quoted 10 lines · Sebastian

Hello all! Hi Jens!

I'd be kind if you would cc me in case you reply as I'm not (yet) subscribed to this list.

Quoted 4 lines · Jens Axboe
Quoted 2 lines · Jens Axboe

Can you try and see how, say, track01 differ? Is it single bytes, chunks of 2352 bytes, or?

Quoted 7 lines
Quoted 3 lines

Oh, and try and disable DMA on the cd driver and repeat your results with ide-cd. It uses DMA, where ide-scsi does not. Dunno what Windows does. It could just be a problem with your drive and DMA enabled rips.

Hi Jens,

I did as you said and disabled dma:

Just a thought, but could you try ripping to something without a header, like RAW? In your case you seem to have been lucky and it'll make no difference, but WAV headers can vary slightly even if the PCM contained within is identical.

Signature · 6 lines
Cheers,
Alistair.

'No sense being pessimistic, it probably wouldn't work anyway.'
Third year Computer Science undergraduate.
1F2 55 South Clerk Street, Edinburgh, UK.
author
#7 lore raw

I'd be kind if you would cc me in case you reply as I'm not (yet) subscribed to this list.

Quoted 9 lines · Alistair John Strachan

Just a thought, but could you try ripping to something without a header, like RAW? In your case you seem to have been lucky and it'll make no difference, but WAV headers can vary slightly even if the PCM contained within is identical.

Signature · 2 lines
Cheers,
Alistair.

I know that this can happen. But it's not the issue here.

Sebastian

Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
#8 lore raw
In reply to Sebastian
Quoted 1 line · Sebastian

Hello all! Hi Jens!

[...]

Quoted 19 lines

I used the wav compare function in EAC.

1. wav ripped by EAC
####################

What happened? Where?
-------------------------------------------------

Different samples       0:04:08.318 - 0:04:08.362
2100 missing samples    0:04:08.359
Different samples       0:04:08.430 - 0:04:08.433
Different samples       0:04:09.348 - 0:04:09.398

2. wav ripped by cdparanoia
###########################

What happened? Where?
-------------------------------------------------

Quoted 8 lines
Different samples       0:04:08.318 - 0:04:08.362
2039 missing samples    0:04:08.414
Different samples       0:04:08.431 - 0:04:08.434
Different samples       0:04:09.349 - 0:04:09.399

I'm sorry if this isn't what you had in mind when you told me to compare the wav files. If it doesn't help what can I do to compare the files to your liking?

It's funny. It looks like the problems happened twice in the more or less same position.

Jerome

author
#9 lore raw
In reply to Jens Axboe

Hello all!

I'd be kind if you would cc me in case you reply as I'm not (yet) subscribed to this list.

Quoted 1 line · Jens Axboe

unno what Windows does.

EAC prefers ASPI, but it falls back to Windows' native SCSI emu layer if ASPI is not installed.

Sebastian

Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
#10 lore raw
In reply to Sebastian

Hi Sebastian,

Quoted 2 lines · Sebastian

The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.

What were you using? cdparanoia? cdda2wav? (Are there actually that many other options on Linux?)

This may well be a userspace problem, where your ripping program doesn't perform enough integrity checks on the data it's just read, and ide-scsi happens to make things slow enough for errors to not occur?

Shooting in the dark,

Signature · 1 line
Joshua Kwan
author
#11 lore raw

Hi all!

Quoted 8 lines · Joshua Kwan

Hi Sebastian,

Quoted 2 lines · Sebastian

The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.

What were you using? cdparanoia? cdda2wav? (Are there actually that many other options on Linux?)

I use cdparanoia.

Quoted 5 lines

This may well be a userspace problem, where your ripping program doesn't perform enough integrity checks on the data it's just read, and ide-scsi happens to make things slow enough for errors to not occur?

I reripped the disc with ide-cd but now with read speed reduced to 2x. Exactly the same md5sums as with ide-cd at full velocity.

Quoted 4 lines

Shooting in the dark,

Signature · 1 line
Joshua Kwan

Sebastian

Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
#12 lore raw
Quoted 11 lines · Sebastian

Hi all!

Quoted 8 lines · Joshua Kwan

Hi Sebastian,

Quoted 2 lines · Sebastian

The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.

What were you using? cdparanoia? cdda2wav? (Are there actually that many other options on Linux?)

I use cdparanoia.

Try cdparanoia -Bvz

This will cause the rip to be extremely careful and make sure everything is exactly right. It works well for me and was recommended by someone I trust. I hop it works for you..

Cheers,
Mark

author
#13 lore raw

Hi everybody,

Quoted 13 lines · Mark Knecht
Quoted 11 lines · Sebastian

Hi all!

Quoted 8 lines · Joshua Kwan

Hi Sebastian,

Quoted 2 lines · Sebastian

The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.

What were you using? cdparanoia? cdda2wav? (Are there actually that many other options on Linux?)

I use cdparanoia.

Quoted 9 lines

Try cdparanoia -Bvz

This will cause the rip to be extremely careful and make sure everything is exactly right. It works well for me and was recommended by someone I trust. I hop it works for you..

Cheers,
Mark

I used cdparanoia -BzX -O48 for every rip.
Just to be clear, I'm not writing to this list because I have problems with an application. :) Rather I like to know where to fix this problem, in kernelland or userspace, like, should I start getting into cdparanoia or reading the o'Reilly book about kernel drivers?

Thanks

Sebastian

Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
#14 lore raw
Quoted 22 lines · Sebastian

Hi everybody,

Quoted 19 lines · Mark Knecht
Quoted 11 lines · Sebastian

Hi all!

Quoted 8 lines · Joshua Kwan

Hi Sebastian,

Quoted 2 lines · Sebastian

The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.

What were you using? cdparanoia? cdda2wav? (Are there actually that many other options on Linux?)

I use cdparanoia.

Try cdparanoia -Bvz

This will cause the rip to be extremely careful and make sure everything is exactly right. It works well for me and was recommended by someone I trust. I hop it works for you..

Quoted 8 lines
Quoted 3 lines

Cheers,
Mark

I used cdparanoia -BzX -O48 for every rip.
Just to be clear, I'm not writing to this list because I have problems with an application. :) Rather I like to know where to fix this problem, in kernelland or userspace, like, should I start getting into cdparanoia or reading the o'Reilly book about kernel drivers?

I missed most of this thread, please don't dump people from the cc list!

Can you put one of the tracks somewhere where I can reach them? Just one of the EAC/ide-scsi ripped vs the ide-cd version. You should probably just privately mail me, we don't want to encourage music piracy :-)

Signature · 1 line
Jens Axboe
#15 lore raw
Quoted 32 lines · Jens Axboe
Quoted 22 lines · Sebastian

Hi everybody,

Quoted 19 lines · Mark Knecht
Quoted 11 lines · Sebastian

Hi all!

Quoted 8 lines · Joshua Kwan

Hi Sebastian,

Quoted 2 lines · Sebastian

The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.

What were you using? cdparanoia? cdda2wav? (Are there actually that many other options on Linux?)

I use cdparanoia.

Try cdparanoia -Bvz

This will cause the rip to be extremely careful and make sure everything is exactly right. It works well for me and was recommended by someone I trust. I hop it works for you..

Quoted 8 lines
Quoted 3 lines

Cheers,
Mark

I used cdparanoia -BzX -O48 for every rip.
Just to be clear, I'm not writing to this list because I have problems with an application. :) Rather I like to know where to fix this problem, in kernelland or userspace, like, should I start getting into cdparanoia or reading the o'Reilly book about kernel drivers?

Quoted 5 lines

I missed most of this thread, please don't dump people from the cc list!

Can you put one of the tracks somewhere where I can reach them? Just one of the EAC/ide-scsi ripped vs the ide-cd version. You should probably just privately mail me, we don't want to encourage music piracy :-)

One more question - when using ide-scsi, does it use the SG_IO ioctl to rip cdda, or does it use CDROMREADAUDIO like I'm assuming it does with ide-cd? Is there a way to force SG_IO usage with a given device in cdparanoia? If so, please try ide-cd with SG_IO usage instead.

Signature · 1 line
Jens Axboe
author
#16 lore raw
Quoted 6 lines · Jens Axboe

One more question - when using ide-scsi, does it use the SG_IO ioctl to rip cdda, or does it use CDROMREADAUDIO like I'm assuming it does with ide-cd? Is there a way to force SG_IO usage with a given device in cdparanoia? If so, please try ide-cd with SG_IO usage instead.

There's one reference in cdparanoia to CDROMREADAUDIO, none at all to SG_IO:

interface/cooked_interface.c line 89:

  do {
    if((err=ioctl(d->ioctl_fd, CDROMREADAUDIO, &arg))){
      if(!d->error_retry)return(-7);
      switch(errno){
      ...

http://svn.xiph.org/trunk/cdparanoia/interface/cooked_interface.c

Sebastian

Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
#17 lore raw
Quoted 12 lines · Sebastian
Quoted 6 lines · Jens Axboe

One more question - when using ide-scsi, does it use the SG_IO ioctl to rip cdda, or does it use CDROMREADAUDIO like I'm assuming it does with ide-cd? Is there a way to force SG_IO usage with a given device in cdparanoia? If so, please try ide-cd with SG_IO usage instead.

There's one reference in cdparanoia to CDROMREADAUDIO, none at all to SG_IO:

interface/cooked_interface.c line 89:

Quoted 7 lines
  do {
    if((err=ioctl(d->ioctl_fd, CDROMREADAUDIO, &arg))){
      if(!d->error_retry)return(-7);
      switch(errno){
      ...

http://svn.xiph.org/trunk/cdparanoia/interface/cooked_interface.c

(please, don't drop me from the cc list!!)

it might be using the older sg interface, opening read/write to /dev/sgX char devices directly. In which case you can't test it with ide-cd, sadly.

Signature · 1 line
Jens Axboe

A message that is not in the archive (sent off-list or not yet received): 20060107112443.GA18749@section_eight.mops.rwth-aachen.de

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): 20060107115449.GB20748@section_eight.mops.rwth-aachen.de

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#22 lore raw

(please, don't drop me from the cc list!!)

Quoted 5 lines · Jens Axboe

(please, don't drop me from the cc list!!)

it might be using the older sg interface, opening read/write to /dev/sgX char devices directly. In which case you can't test it with ide-cd, sadly.

You wrote about accessing the drive with SG_IO while using ide-cd. So it is possible to use scsi commands though using ide-cd? I can't find any documentation on that, though. Could you point me towards it? I can try to adapt cdparanoia.

Cheers

Sebastian

Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
#23 lore raw
Quoted 5 lines · Sebastian

(please, don't drop me from the cc list!!)

Quoted 2 lines · Jens Axboe

(please, don't drop me from the cc list!!)

Quoted 8 lines
Quoted 3 lines

it might be using the older sg interface, opening read/write to /dev/sgX char devices directly. In which case you can't test it with ide-cd, sadly.

You wrote about accessing the drive with SG_IO while using ide-cd. So it is possible to use scsi commands though using ide-cd? I can't find any documentation on that, though. Could you point me towards it? I can try to adapt cdparanoia.

You can use SG_IO on any block device that accepts "SCSI" commands in the end, like ide-cd. Google for the sg driver documentation and you will find there are various ways to submit sg commands. cdparanoia likely uses the old variant of opening the char device and read/writing commands to it, if you convert it to SG_IO it could use that transport always (on 2.6 kernels and newer).

Signature · 1 line
Jens Axboe
author
#24 lore raw

(please, don't drop me from the cc list!!)

Yay, problem solved!

Quoted 11 lines · Jens Axboe

You can use SG_IO on any block device that accepts "SCSI" commands in the end, like ide-cd. Google for the sg driver documentation and you will find there are various ways to submit sg commands. cdparanoia likely uses the old variant of opening the char device and read/writing commands to it, if you convert it to SG_IO it could use that transport always (on 2.6 kernels and newer).

Signature · 1 line
Jens Axboe

Thanks guys for all your pointers. I googled for cdparanoia and SG_IO and found out that someone at Red Hat already patched cdparanoia to use SG_IO. http://people.redhat.com/pjones/cdparanoia/

If you're interested, the patches can be found in the src rpm:
http://people.redhat.com/pjones/cdparanoia/thomasvs/cdparanoia-alpha9.8-27.src.rpm

Well, I'm gonna submit an updated ebuild for Gentoo. Let's see how this thing evolves.

Thanks again for all your help, especially Jens!

Cheers

Sebastian

P.S.: I already reripped the test disc using ide-cd and the SG_IO-patched cdparanoia and the results are perfect. OMG, I bought Win XP Home 2 months ago because of this (so I can use Exact Audio Copy). I guess I can remove XP from my drive now and sell it to some wretched guy :) Harhar.

S.

Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
#25 lore raw
Quoted 6 lines · Sebastian

Yay, problem solved!

Quoted 7 lines

P.S.: I already reripped the test disc using ide-cd and the SG_IO-patched cdparanoia and the results are perfect. OMG, I bought Win XP Home 2 months ago because of this (so I can use Exact Audio Copy). I guess I can remove XP from my drive now and sell it to some wretched guy :) Harhar.

S.

Yes, but now we need to find out why one interface fails while another works.. I have the same problem here using cdrdao when ripping entire disk images. I'd love to fix the real issue rather than work around it by having userspace use another interface. I would have thought that both interfaces should return the same data..

Brad

Signature · 3 lines
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams
author
#26 lore raw

(please, don't forget to me and [email protected])

Quoted 15 lines · Brad Campbell
Quoted 6 lines · Sebastian

Yay, problem solved!

Quoted 7 lines

P.S.: I already reripped the test disc using ide-cd and the SG_IO-patched cdparanoia and the results are perfect. OMG, I bought Win XP Home 2 months ago because of this (so I can use Exact Audio Copy). I guess I can remove XP from my drive now and sell it to some wretched guy :) Harhar.

S.

Quoted 7 lines

Yes, but now we need to find out why one interface fails while another works.. I have the same problem here using cdrdao when ripping entire disk images. I'd love to fix the real issue rather than work around it by having userspace use another interface.
I would have thought that both interfaces should return the same data..

Brad

I think cdrdao can use SG_IO if you tell it to. Check their documentation. Or did I misunderstand what you're saying?

Sebastian

Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
#27 lore raw
Quoted 1 line · Sebastian

(please, don't forget to me and [email protected])

The list convention regarding cc's is quite simple. ALWAYS use reply-to-all when responding to LKML mail. Therefore "please cc me, I'm not subscribed" is completely redundant.

Lee

#28 lore raw
In reply to Sebastian
Quoted 1 line · Sebastian

(please, don't forget to me and [email protected])

Did I? I'm sorry, I was *sure* I hit reply-to-all.

Quoted 6 lines
Quoted 4 lines

Yes, but now we need to find out why one interface fails while another works.. I have the same problem here using cdrdao when ripping entire disk images. I'd love to fix the real issue rather than work around it by having userspace use another interface. I would have thought that both interfaces should return the same data..

Brad

I think cdrdao can use SG_IO if you tell it to. Check their documentation. Or did I misunderstand what you're saying?

Slightly.. If I'm not mistaken, we have a piece of software (for arguments sake let's call it cdparanoia). The stock software uses the ATA or ATAPI interface and produces bodged reads, the modified version you have uses SG_IO and produces accurate reads.

What I'm wondering is why the difference, and is there a problem with the ATA/ATAPI interface that leads to this that needs looking into.

*or* is it a userspace problem with the way cdparanoia is using that interface

*or* is it an inherent problem/limitation with that particular interface

I can reproduce this with cdrdao by reading a cd 10 times with all combinations of paranoia mode attempted. I get 10 x ~550 meg data.bin files that all differ.

I'm thinking I need to work up a script that diffs them by audio sector size chunks to see if there is a pattern there somewhere, but you would think that somewhere 2 reads in 10 would be identical.

Brad

Signature · 3 lines
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams
#29 lore raw
Quoted 13 lines · Brad Campbell
Quoted 1 line · Sebastian

(please, don't forget to me and [email protected])

Did I? I'm sorry, I was *sure* I hit reply-to-all.

Quoted 8 lines
Quoted 7 lines

Yes, but now we need to find out why one interface fails while another works.. I have the same problem here using cdrdao when ripping entire disk images. I'd love to fix the real issue rather than work around it by having userspace use another interface.
I would have thought that both interfaces should return the same data..

Brad

Quoted 7 lines
Quoted 2 lines

I think cdrdao can use SG_IO if you tell it to. Check their documentation. Or did I misunderstand what you're saying?

Slightly.. If I'm not mistaken, we have a piece of software (for arguments sake let's call it cdparanoia). The stock software uses the ATA or ATAPI interface and produces bodged reads, the modified version you have uses SG_IO and produces accurate reads.

It's the cdrom CDROMREADAUDIO vs SG_IO interface, there's not ata/atapi specific interface.

Quoted 2 lines

What I'm wondering is why the difference, and is there a problem with the ATA/ATAPI interface that leads to this that needs looking into.

Definitely, that's the big question. It could just be that cdparanoia enables extra error correction/checking, which it can do since it tailors the SCSI commands to its liking. Where as with CDROMREADAUDIO, you're basically stuck with whatever is coded into the cdrom layer.

Quoted 1 line

*or* is it an inherent problem/limitation with that particular interface

Could very well be. Once could use blktrace to see what the commands submitted to the drives look like and get some data out of that.

Sebastian, care to try one more thing? Patch your kernel with this little patch and try ripping a known "faulty" track again _not_ using SG_IO. See if that produces the same faulty results again, or if it actually works.

drivers/cdrom/cdrom.c +1 -1
index 1539603..2e44d81 100644
@@ -426,7 +426,7 @@ int register_cdrom(struct cdrom_device_i
 		cdi->exit = cdrom_mrw_exit;
 
 	if (cdi->disk)
-		cdi->cdda_method = CDDA_BPC_FULL;
+		cdi->cdda_method = CDDA_BPC_SINGLE;
 	else
 		cdi->cdda_method = CDDA_OLD;
 
Signature · 1 line
Jens Axboe
author
#30 lore raw
Quoted 22 lines · Jens Axboe

Sebastian, care to try one more thing? Patch your kernel with this little patch and try ripping a known "faulty" track again _not_ using SG_IO. See if that produces the same faulty results again, or if it actually works.

drivers/cdrom/cdrom.c +1 -1
index 1539603..2e44d81 100644
@@ -426,7 +426,7 @@ int register_cdrom(struct cdrom_device_i
 		cdi->exit = cdrom_mrw_exit;
 
 	if (cdi->disk)
-		cdi->cdda_method = CDDA_BPC_FULL;
+		cdi->cdda_method = CDDA_BPC_SINGLE;
 	else
 		cdi->cdda_method = CDDA_OLD;
 
Signature · 2 lines
-- 
Jens Axboe

Hi Jens,

I applied your patch, recompiled the kernel, rebooted and recompiled cdparanoia without the Red Hat patches. Extracting the first track of my test cd the result was the same as without the kernel patch with ide-cd using the cooked interface (md5 e8319ccc20d053557578b9ca3eb368dd).

Sorry :)

Sebastian

Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
#31 lore raw
Quoted 25 lines · Sebastian
Quoted 22 lines · Jens Axboe

Sebastian, care to try one more thing? Patch your kernel with this little patch and try ripping a known "faulty" track again _not_ using SG_IO. See if that produces the same faulty results again, or if it actually works.

drivers/cdrom/cdrom.c +1 -1
index 1539603..2e44d81 100644
@@ -426,7 +426,7 @@ int register_cdrom(struct cdrom_device_i
 		cdi->exit = cdrom_mrw_exit;
 
 	if (cdi->disk)
-		cdi->cdda_method = CDDA_BPC_FULL;
+		cdi->cdda_method = CDDA_BPC_SINGLE;
 	else
 		cdi->cdda_method = CDDA_OLD;
 
Signature · 2 lines
-- 
Jens Axboe

Hi Jens,

Quoted 6 lines

I applied your patch, recompiled the kernel, rebooted and recompiled cdparanoia without the Red Hat patches. Extracting the first track of my test cd the result was the same as without the kernel patch with ide-cd using the cooked interface (md5 e8319ccc20d053557578b9ca3eb368dd).

Sorry :)

Well it's actually a good thing, because then at least it's not a bug with the multi-frame extraction. So my guess would still be at the error correction possibilities that the application has, in which case CDROMREADAUDIO is just an inferior interface for this sort of thing. It also doesn't give the issuer a chance to look at potential error reasons, since the extended status isn't available.

Signature · 1 line
Jens Axboe
#32 lore raw
Quoted 6 lines · Jens Axboe

Well it's actually a good thing, because then at least it's not a bug with the multi-frame extraction. So my guess would still be at the error correction possibilities that the application has, in which case CDROMREADAUDIO is just an inferior interface for this sort of thing. It also doesn't give the issuer a chance to look at potential error reasons, since the extended status isn't available.

For what it's worth; when allowing only flawless rips with cdparanoia (*) standard cdparanoia 9.8 and the SG_IO patched version of the same give me the exact same rips always, on my ATAPI Plextor "PlexWriter Premium", a drive that's specifically good at CDDA.

One thing that _is_ different is that the SG_IO version very frequently (soft) locks up the machine, with:

===
hdc: DMA timeout retry
hdc: timeout waiting for DMA
hdc: status error: status=0x7f { DriveReady DeviceFault SeekComplete DataRequest CorrectedError Index Error }
hdc: status error: error=0x7f { IllegalLengthIndication EndOfMedia AbortedCommand MediaChangeRequested LastFailedSense=0x07 }
ide: failed opcode was: unknown
hdc: drive not ready for command
hdc: ATAPI reset complete
hdc: irq timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: ATAPI reset complete
hdc: irq timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: status timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: drive not ready for command
hdc: status timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: DMA disabled
hdc: drive not ready for command
hdc: ATAPI reset complete
hdc: irq timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: ATAPI reset complete
hdc: irq timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: status timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: drive not ready for command

BUG: soft lockup detected on CPU#0!

Pid: 0, comm: swapper
EIP: 0060:[<c01d111f>] CPU: 0
EIP is at ide_inb+0x3/0x7
EFLAGS: 00000206 Not tainted (2.6.15-local-wc)
EAX: 00000180 EBX: c0329184 ECX: 2cb065ff EDX: 00000177
ESI: 00000282 EDI: 001403db EBP: c03290f0 DS: 007b ES: 007b
CR0: 8005003b CR2: b7c5d004 CR3: 1fd1f000 CR4: 000006d0

 [<c01d16ca>] ide_wait_stat+0xa0/0xfd
 [<c01d07cb>] start_request+0x10c/0x1a7
 [<c01d0b42>] ide_do_request+0x2bb/0x2f9
 [<c01d00c5>] ide_atapi_error+0x53/0x78
 [<c01da977>] cdrom_newpc_intr+0x0/0x2fd
 [<c01d0dfd>] ide_timer_expiry+0x195/0x1bf
 [<c0118ebe>] run_timer_softirq+0x140/0x181
 [<c01d0c68>] ide_timer_expiry+0x0/0x1bf
 [<c0115b01>] __do_softirq+0x35/0x7d
 [<c0103c54>] do_softirq+0x38/0x3f
 =======================
 [<c0115bda>] irq_exit+0x29/0x34
 [<c0103b84>] do_IRQ+0x46/0x4e
 [<c01026ce>] common_interrupt+0x1a/0x20
 [<c0100a73>] default_idle+0x2b/0x53
 [<c0100af0>] cpu_idle+0x41/0x63
 [<c02ea621>] start_kernel+0x13d/0x13f

hdc: status timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: drive not ready for command
hdc: status timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
===

Ad infitum. Needless to say, I switched back to regular (unpatched from upstream) cdparanoia which doesn't show this problem. The patching I did was from this message earlier in this thread:

http://marc.theaimsgroup.com/?l=linux-kernel&m=113665002702563&w=2

First I tried with only the .sgio and .labels patches from that src.rpm applied, saw the problem, then retried with all of redhat's patches applied (just in case) but did not see an improvement.

As said, when things do work, the rips are exactly the same as the rips made with the standard cdparanoia.

(*) by "allowing only flawless rips" I mean using "cdparanoia -v -z -B" and only ever allowing spaces in the progressbar cdparanoia leaves behind as it goes along. When anything else (-, +, !, ...) appears, I re-rip the entire track, or the bit in which the error occured, until I have a "spaces only" copy. Two of such rips are always the same while on the other hand a non-spaces-only rip generally differs from any other rip made of that same track.

Rene.

#33 lore raw
Quoted 2 lines · Rene Herman

One thing that _is_ different is that the SG_IO version very frequently (soft) locks up the machine, with:

So you are saying that once you see the BUG: softlockup message the system is unresponsive and must be rebooted?

Lee

#34 lore raw
Quoted 6 lines · Lee Revell
Quoted 2 lines

One thing that _is_ different is that the SG_IO version very frequently (soft) locks up the machine, with:

So you are saying that once you see the BUG: softlockup message the system is unresponsive and must be rebooted?

Well, a bit before that -- it takes a while for the softlockup timer to trigger. Yes and no, the machine's 'dead' while this is happening but in the end (minutes) it does recover in so far that it's no longer trying (but does leave around a cdparanoia in D state). Yes, I reboot, and then
things work again, until... well, they do not, which is quickly with
the SG_IO patched cdparanoia.

It's probably more useful to test with a very minimal CDDA extraction tool using SG_IO, if Jens has any such tool lying around, and if anyone _wants_ me to test. As said, I personally just switched back to using regular cdparanoia, which is working fine for me.

Rene.

#36 lore raw
In reply to Sebastian
Quoted 4 lines · Sebastian

You wrote about accessing the drive with SG_IO while using ide-cd. So it is possible to use scsi commands though using ide-cd? I can't find any documentation on that, though. Could you point me towards it? I can try to adapt cdparanoia.

In 2.6 yes. The SG_IO ioctl works on any block device fd. The commands you can issue then depend upon the mode of opening. Some commands require root, generally safe ones read or write access.

Due to bugs still present in the block layer its also the only way to reliably copy a data CD as well.

Alan