Digital Audio Extraction with ATAPI drives far from perfect
Hello all!
I'd be kind if you would cc me in case you reply as I'm not (yet) subscribed to this list.
I'm a music buff :) Well, I guess most people are in one way or the other. But in addition to that I like bitperfect ripping very much. Since some time this doesn't work well with linux anymore.
I'd like to know if there's something in the making already or what could be done to solve this issue. A supporting layer for the ide system, a new driver maybe? I think this can't be fixed in userspace, right?
I ripped an audio disc in different ways and compared the results using md5sum. As you can see the wav data is perfect when using ide-scsi emulation. On the contrary, using ide lead to errors.
Linux version 2.6.15 (root@section_eight) (gcc-Version 3.4.4 (Gentoo 3.4.4-r1, ssp-3.4.4-1.0, pie-8.7.8))
In all cases I used the same drive, a NEC ND-4550A dvd writer.
The first series of wav files was ripped with EAC in Windows and is bitperfect:
8cab5ca4820a753ebb3cb7e3c5c34e6a 01.Man In A Suitcase.wav 187f90f900cffd36ce56e97a3bcf595e 02.Box Of Six.wav 8cc168bb50e80a06693a01a9cd34bbdf 03.Mysterons.wav e82169e5ea1b441b80db96fce12fd109 04.Justified.wav 8d807b7ac19f90049aec6ff177e9b486 05.Department S.wav 130306e9a564c844d5269256f38afca7 06.Area Code 51.wav 96489dbfcab8f97e7e450cf8db2c8aaa 07.Has To Be.wav 4597a6ed75e201916a3479f05eb86405 08.No. 5.wav f978327a98fc6359be2fc25eb865211d 09.Among The Cybermen.wav e316e140b4b0cd66e5822edae22dadb1 10.Unspeakable Elvis.wav 3792a680b1ba729de9185043d331186f 11.Xodiak.wav ba534fd8eb42dd84aa7b59ab3ae6f132 12.Northern Wisdom.wav d6346ab76696dddf735a5b752aa7888b 13.Trinity Road.wav
The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.
The third series was done with ide-cd. Erroneous data is marked with a (!):
e8319ccc20d053557578b9ca3eb368dd track01.cdda.wav (!) cb978f86ddc18c9df1b7e91705380bc5 track02.cdda.wav (!) 35f1b296d72a8708d03aeb540a3b4f30 track03.cdda.wav (!) e82169e5ea1b441b80db96fce12fd109 track04.cdda.wav 8d807b7ac19f90049aec6ff177e9b486 track05.cdda.wav 02561939763d67aacf23157c09966a89 track06.cdda.wav (!) 9724b0a3e2295084613da9df7397ae6d track07.cdda.wav (!) c2d85b3d10428aad66664d0fb3e4c71a track08.cdda.wav (!) 5116b2fae44b8b86fbf40b9bac9a8268 track09.cdda.wav (!) 9e6a5ab2dab76e1677667f586895293a track10.cdda.wav (!) 3792a680b1ba729de9185043d331186f track11.cdda.wav ba534fd8eb42dd84aa7b59ab3ae6f132 track12.cdda.wav d6346ab76696dddf735a5b752aa7888b track13.cdda.wav
Sebastian
Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
Quoted 45 lines · Sebastian
Hello all!
I'd be kind if you would cc me in case you reply as I'm not (yet) subscribed to this list.
I'm a music buff :) Well, I guess most people are in one way or the other. But in addition to that I like bitperfect ripping very much. Since some time this doesn't work well with linux anymore.
I'd like to know if there's something in the making already or what could be done to solve this issue. A supporting layer for the ide system, a new driver maybe? I think this can't be fixed in userspace, right?
I ripped an audio disc in different ways and compared the results using md5sum. As you can see the wav data is perfect when using ide-scsi emulation. On the contrary, using ide lead to errors.
Linux version 2.6.15 (root@section_eight) (gcc-Version 3.4.4 (Gentoo 3.4.4-r1, ssp-3.4.4-1.0, pie-8.7.8))
In all cases I used the same drive, a NEC ND-4550A dvd writer.
The first series of wav files was ripped with EAC in Windows and is bitperfect:
8cab5ca4820a753ebb3cb7e3c5c34e6a 01.Man In A Suitcase.wav 187f90f900cffd36ce56e97a3bcf595e 02.Box Of Six.wav 8cc168bb50e80a06693a01a9cd34bbdf 03.Mysterons.wav e82169e5ea1b441b80db96fce12fd109 04.Justified.wav 8d807b7ac19f90049aec6ff177e9b486 05.Department S.wav 130306e9a564c844d5269256f38afca7 06.Area Code 51.wav 96489dbfcab8f97e7e450cf8db2c8aaa 07.Has To Be.wav 4597a6ed75e201916a3479f05eb86405 08.No. 5.wav f978327a98fc6359be2fc25eb865211d 09.Among The Cybermen.wav e316e140b4b0cd66e5822edae22dadb1 10.Unspeakable Elvis.wav 3792a680b1ba729de9185043d331186f 11.Xodiak.wav ba534fd8eb42dd84aa7b59ab3ae6f132 12.Northern Wisdom.wav d6346ab76696dddf735a5b752aa7888b 13.Trinity Road.wavThe second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.
The third series was done with ide-cd. Erroneous data is marked with a (!):
Quoted 13 lines
e8319ccc20d053557578b9ca3eb368dd track01.cdda.wav (!) cb978f86ddc18c9df1b7e91705380bc5 track02.cdda.wav (!) 35f1b296d72a8708d03aeb540a3b4f30 track03.cdda.wav (!) e82169e5ea1b441b80db96fce12fd109 track04.cdda.wav 8d807b7ac19f90049aec6ff177e9b486 track05.cdda.wav 02561939763d67aacf23157c09966a89 track06.cdda.wav (!) 9724b0a3e2295084613da9df7397ae6d track07.cdda.wav (!) c2d85b3d10428aad66664d0fb3e4c71a track08.cdda.wav (!) 5116b2fae44b8b86fbf40b9bac9a8268 track09.cdda.wav (!) 9e6a5ab2dab76e1677667f586895293a track10.cdda.wav (!) 3792a680b1ba729de9185043d331186f track11.cdda.wav ba534fd8eb42dd84aa7b59ab3ae6f132 track12.cdda.wav d6346ab76696dddf735a5b752aa7888b track13.cdda.wav
Can you try and see how, say, track01 differ? Is it single bytes, chunks of 2352 bytes, or?
Signature · 1 line
Jens Axboe
Quoted 60 lines · Jens Axboe
Quoted 45 lines · Sebastian
Hello all!
I'd be kind if you would cc me in case you reply as I'm not (yet) subscribed to this list.
I'm a music buff :) Well, I guess most people are in one way or the other. But in addition to that I like bitperfect ripping very much. Since some time this doesn't work well with linux anymore.
I'd like to know if there's something in the making already or what could be done to solve this issue. A supporting layer for the ide system, a new driver maybe? I think this can't be fixed in userspace, right?
I ripped an audio disc in different ways and compared the results using md5sum. As you can see the wav data is perfect when using ide-scsi emulation. On the contrary, using ide lead to errors.
Linux version 2.6.15 (root@section_eight) (gcc-Version 3.4.4 (Gentoo 3.4.4-r1, ssp-3.4.4-1.0, pie-8.7.8))
In all cases I used the same drive, a NEC ND-4550A dvd writer.
The first series of wav files was ripped with EAC in Windows and is bitperfect:
8cab5ca4820a753ebb3cb7e3c5c34e6a 01.Man In A Suitcase.wav 187f90f900cffd36ce56e97a3bcf595e 02.Box Of Six.wav 8cc168bb50e80a06693a01a9cd34bbdf 03.Mysterons.wav e82169e5ea1b441b80db96fce12fd109 04.Justified.wav 8d807b7ac19f90049aec6ff177e9b486 05.Department S.wav 130306e9a564c844d5269256f38afca7 06.Area Code 51.wav 96489dbfcab8f97e7e450cf8db2c8aaa 07.Has To Be.wav 4597a6ed75e201916a3479f05eb86405 08.No. 5.wav f978327a98fc6359be2fc25eb865211d 09.Among The Cybermen.wav e316e140b4b0cd66e5822edae22dadb1 10.Unspeakable Elvis.wav 3792a680b1ba729de9185043d331186f 11.Xodiak.wav ba534fd8eb42dd84aa7b59ab3ae6f132 12.Northern Wisdom.wav d6346ab76696dddf735a5b752aa7888b 13.Trinity Road.wavThe second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.
The third series was done with ide-cd. Erroneous data is marked with a (!):
Quoted 13 lines
e8319ccc20d053557578b9ca3eb368dd track01.cdda.wav (!) cb978f86ddc18c9df1b7e91705380bc5 track02.cdda.wav (!) 35f1b296d72a8708d03aeb540a3b4f30 track03.cdda.wav (!) e82169e5ea1b441b80db96fce12fd109 track04.cdda.wav 8d807b7ac19f90049aec6ff177e9b486 track05.cdda.wav 02561939763d67aacf23157c09966a89 track06.cdda.wav (!) 9724b0a3e2295084613da9df7397ae6d track07.cdda.wav (!) c2d85b3d10428aad66664d0fb3e4c71a track08.cdda.wav (!) 5116b2fae44b8b86fbf40b9bac9a8268 track09.cdda.wav (!) 9e6a5ab2dab76e1677667f586895293a track10.cdda.wav (!) 3792a680b1ba729de9185043d331186f track11.cdda.wav ba534fd8eb42dd84aa7b59ab3ae6f132 track12.cdda.wav d6346ab76696dddf735a5b752aa7888b track13.cdda.wav
Quoted 2 lines
Can you try and see how, say, track01 differ? Is it single bytes, chunks of 2352 bytes, or?
Oh, and try and disable DMA on the cd driver and repeat your results with ide-cd. It uses DMA, where ide-scsi does not. Dunno what Windows does. It could just be a problem with your drive and DMA enabled rips.
Signature · 1 line
Jens Axboe
Jens Axboe wrote:
...
Quoted 2 lines
Oh, and try and disable DMA on the cd driver and repeat your results with ide-cd. It uses DMA, where ide-scsi does not.
Eh? Sure it does!
Quoted 6 lines · Mark Lord
Jens Axboe wrote:
...Quoted 2 lines
Oh, and try and disable DMA on the cd driver and repeat your results with ide-cd. It uses DMA, where ide-scsi does not.
Eh? Sure it does!
Only for transfers multiple of 1k, which the DMA ripping is not.
Signature · 1 line
Jens Axboe
Hello all! Hi Jens!
I'd be kind if you would cc me in case you reply as I'm not (yet) subscribed to this list.
Quoted 8 lines · Jens Axboe
Quoted 3 lines · Jens Axboe
Can you try and see how, say, track01 differ? Is it single bytes, chunks of 2352 bytes, or?
Oh, and try and disable DMA on the cd driver and repeat your results with ide-cd. It uses DMA, where ide-scsi does not. Dunno what Windows does. It could just be a problem with your drive and DMA enabled rips.
Hi Jens,
I did as you said and disabled dma:
/dev/hdc:
IO_support = 1 (32-bit) unmaskirq = 1 (on) using_dma = 0 (off) keepsettings = 0 (off) readonly = 0 (off)
readahead = 256 (on)
HDIO_GETGEO failed: Invalid argument
Then I reripped the whole disc. Now this is remarkable. The rip still has errors, but the errors are not only in the same tracks. They are exactly the same errors (md5sums are equal to the ones I yielded from using ide-cd before):
e8319ccc20d053557578b9ca3eb368dd track01.cdda.wav (!) cb978f86ddc18c9df1b7e91705380bc5 track02.cdda.wav (!) 35f1b296d72a8708d03aeb540a3b4f30 track03.cdda.wav (!) e82169e5ea1b441b80db96fce12fd109 track04.cdda.wav 8d807b7ac19f90049aec6ff177e9b486 track05.cdda.wav 02561939763d67aacf23157c09966a89 track06.cdda.wav (!) 9724b0a3e2295084613da9df7397ae6d track07.cdda.wav (!) c2d85b3d10428aad66664d0fb3e4c71a track08.cdda.wav (!) 5116b2fae44b8b86fbf40b9bac9a8268 track09.cdda.wav (!) 9e6a5ab2dab76e1677667f586895293a track10.cdda.wav (!) 3792a680b1ba729de9185043d331186f track11.cdda.wav ba534fd8eb42dd84aa7b59ab3ae6f132 track12.cdda.wav d6346ab76696dddf735a5b752aa7888b track13.cdda.wav
I used the wav compare function in EAC.
1. wav ripped by EAC
####################
What happened? Where?
-------------------------------------------------
Different samples 0:04:08.318 - 0:04:08.362 2100 missing samples 0:04:08.359 Different samples 0:04:08.430 - 0:04:08.433 Different samples 0:04:09.348 - 0:04:09.398
2. wav ripped by cdparanoia
###########################
What happened? Where?
-------------------------------------------------
Different samples 0:04:08.318 - 0:04:08.362 2039 missing samples 0:04:08.414 Different samples 0:04:08.431 - 0:04:08.434 Different samples 0:04:09.349 - 0:04:09.399
I'm sorry if this isn't what you had in mind when you told me to compare the wav files. If it doesn't help what can I do to compare the files to your liking?
Sebastian
Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
Quoted 10 lines · Sebastian
Hello all! Hi Jens!
I'd be kind if you would cc me in case you reply as I'm not (yet) subscribed to this list.
Quoted 4 lines · Jens Axboe
Quoted 2 lines · Jens Axboe
Can you try and see how, say, track01 differ? Is it single bytes, chunks of 2352 bytes, or?
Quoted 7 lines
Quoted 3 lines
Oh, and try and disable DMA on the cd driver and repeat your results with ide-cd. It uses DMA, where ide-scsi does not. Dunno what Windows does. It could just be a problem with your drive and DMA enabled rips.
Hi Jens,
I did as you said and disabled dma:
Just a thought, but could you try ripping to something without a header, like RAW? In your case you seem to have been lucky and it'll make no difference, but WAV headers can vary slightly even if the PCM contained within is identical.
Signature · 6 lines
Cheers, Alistair. 'No sense being pessimistic, it probably wouldn't work anyway.' Third year Computer Science undergraduate. 1F2 55 South Clerk Street, Edinburgh, UK.
I'd be kind if you would cc me in case you reply as I'm not (yet) subscribed to this list.
Quoted 9 lines · Alistair John Strachan
Just a thought, but could you try ripping to something without a header, like RAW? In your case you seem to have been lucky and it'll make no difference, but WAV headers can vary slightly even if the PCM contained within is identical.
Signature · 2 lines
Cheers, Alistair.
I know that this can happen. But it's not the issue here.
Sebastian
Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
Quoted 1 line · Sebastian
Hello all! Hi Jens!
[...]
Quoted 19 lines
I used the wav compare function in EAC.
1. wav ripped by EAC
####################What happened? Where?
-------------------------------------------------Different samples 0:04:08.318 - 0:04:08.362 2100 missing samples 0:04:08.359 Different samples 0:04:08.430 - 0:04:08.433 Different samples 0:04:09.348 - 0:04:09.3982. wav ripped by cdparanoia
###########################What happened? Where?
-------------------------------------------------
Quoted 8 lines
Different samples 0:04:08.318 - 0:04:08.362 2039 missing samples 0:04:08.414 Different samples 0:04:08.431 - 0:04:08.434 Different samples 0:04:09.349 - 0:04:09.399I'm sorry if this isn't what you had in mind when you told me to compare the wav files. If it doesn't help what can I do to compare the files to your liking?
It's funny. It looks like the problems happened twice in the more or less same position.
Jerome
Hello all!
I'd be kind if you would cc me in case you reply as I'm not (yet) subscribed to this list.
Quoted 1 line · Jens Axboe
unno what Windows does.
EAC prefers ASPI, but it falls back to Windows' native SCSI emu layer if ASPI is not installed.
Sebastian
Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
Hi Sebastian,
Quoted 2 lines · Sebastian
The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.
What were you using? cdparanoia? cdda2wav? (Are there actually that many other options on Linux?)
This may well be a userspace problem, where your ripping program doesn't perform enough integrity checks on the data it's just read, and ide-scsi happens to make things slow enough for errors to not occur?
Shooting in the dark,
Signature · 1 line
Joshua Kwan
Hi all!
Quoted 8 lines · Joshua Kwan
Hi Sebastian,
Quoted 2 lines · Sebastian
The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.
What were you using? cdparanoia? cdda2wav? (Are there actually that many other options on Linux?)
I use cdparanoia.
Quoted 5 lines
This may well be a userspace problem, where your ripping program doesn't perform enough integrity checks on the data it's just read, and ide-scsi happens to make things slow enough for errors to not occur?
I reripped the disc with ide-cd but now with read speed reduced to 2x. Exactly the same md5sums as with ide-cd at full velocity.
Quoted 4 lines
Shooting in the dark,
Signature · 1 line
Joshua Kwan
Sebastian
Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
Quoted 11 lines · Sebastian
Hi all!
Quoted 8 lines · Joshua Kwan
Hi Sebastian,
Quoted 2 lines · Sebastian
The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.
What were you using? cdparanoia? cdda2wav? (Are there actually that many other options on Linux?)
I use cdparanoia.
Try cdparanoia -Bvz
This will cause the rip to be extremely careful and make sure everything is exactly right. It works well for me and was recommended by someone I trust. I hop it works for you..
Cheers,
Mark
Hi everybody,
Quoted 13 lines · Mark Knecht
Quoted 11 lines · Sebastian
Hi all!
Quoted 8 lines · Joshua Kwan
Hi Sebastian,
Quoted 2 lines · Sebastian
The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.
What were you using? cdparanoia? cdda2wav? (Are there actually that many other options on Linux?)
I use cdparanoia.
Quoted 9 lines
Try cdparanoia -Bvz
This will cause the rip to be extremely careful and make sure everything is exactly right. It works well for me and was recommended by someone I trust. I hop it works for you..
Cheers,
Mark
I used cdparanoia -BzX -O48 for every rip.
Just to be clear, I'm not writing to this list because I have problems with an application. :) Rather I like to know where to fix this problem, in kernelland or userspace, like, should I start getting into cdparanoia or reading the o'Reilly book about kernel drivers?
Thanks
Sebastian
Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
Quoted 22 lines · Sebastian
Hi everybody,
Quoted 19 lines · Mark Knecht
Quoted 11 lines · Sebastian
Hi all!
Quoted 8 lines · Joshua Kwan
Hi Sebastian,
Quoted 2 lines · Sebastian
The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.
What were you using? cdparanoia? cdda2wav? (Are there actually that many other options on Linux?)
I use cdparanoia.
Try cdparanoia -Bvz
This will cause the rip to be extremely careful and make sure everything is exactly right. It works well for me and was recommended by someone I trust. I hop it works for you..
Quoted 8 lines
Quoted 3 lines
Cheers,
MarkI used cdparanoia -BzX -O48 for every rip.
Just to be clear, I'm not writing to this list because I have problems with an application. :) Rather I like to know where to fix this problem, in kernelland or userspace, like, should I start getting into cdparanoia or reading the o'Reilly book about kernel drivers?
I missed most of this thread, please don't dump people from the cc list!
Can you put one of the tracks somewhere where I can reach them? Just one of the EAC/ide-scsi ripped vs the ide-cd version. You should probably just privately mail me, we don't want to encourage music piracy :-)
Signature · 1 line
Jens Axboe
Quoted 32 lines · Jens Axboe
Quoted 22 lines · Sebastian
Hi everybody,
Quoted 19 lines · Mark Knecht
Quoted 11 lines · Sebastian
Hi all!
Quoted 8 lines · Joshua Kwan
Hi Sebastian,
Quoted 2 lines · Sebastian
The second series was ripped with deprecated ide-scsi emulation and yielded the same results as EAC.
What were you using? cdparanoia? cdda2wav? (Are there actually that many other options on Linux?)
I use cdparanoia.
Try cdparanoia -Bvz
This will cause the rip to be extremely careful and make sure everything is exactly right. It works well for me and was recommended by someone I trust. I hop it works for you..
Quoted 8 lines
Quoted 3 lines
Cheers,
MarkI used cdparanoia -BzX -O48 for every rip.
Just to be clear, I'm not writing to this list because I have problems with an application. :) Rather I like to know where to fix this problem, in kernelland or userspace, like, should I start getting into cdparanoia or reading the o'Reilly book about kernel drivers?
Quoted 5 lines
I missed most of this thread, please don't dump people from the cc list!
Can you put one of the tracks somewhere where I can reach them? Just one of the EAC/ide-scsi ripped vs the ide-cd version. You should probably just privately mail me, we don't want to encourage music piracy :-)
One more question - when using ide-scsi, does it use the SG_IO ioctl to rip cdda, or does it use CDROMREADAUDIO like I'm assuming it does with ide-cd? Is there a way to force SG_IO usage with a given device in cdparanoia? If so, please try ide-cd with SG_IO usage instead.
Signature · 1 line
Jens Axboe
Quoted 6 lines · Jens Axboe
One more question - when using ide-scsi, does it use the
SG_IOioctl to rip cdda, or does it use CDROMREADAUDIO like I'm assuming it does with ide-cd? Is there a way to forceSG_IOusage with a given device in cdparanoia? If so, please try ide-cd withSG_IOusage instead.
There's one reference in cdparanoia to CDROMREADAUDIO, none at all to SG_IO:
interface/cooked_interface.c line 89:
do {
if((err=ioctl(d->ioctl_fd, CDROMREADAUDIO, &arg))){
if(!d->error_retry)return(-7);
switch(errno){
...http://svn.xiph.org/trunk/cdparanoia/interface/cooked_interface.c
Sebastian
Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
Quoted 12 lines · Sebastian
Quoted 6 lines · Jens Axboe
One more question - when using ide-scsi, does it use the
SG_IOioctl to rip cdda, or does it use CDROMREADAUDIO like I'm assuming it does with ide-cd? Is there a way to forceSG_IOusage with a given device in cdparanoia? If so, please try ide-cd withSG_IOusage instead.There's one reference in cdparanoia to CDROMREADAUDIO, none at all to
SG_IO:
interface/cooked_interface.cline 89:
Quoted 7 lines
do { if((err=ioctl(d->ioctl_fd, CDROMREADAUDIO, &arg))){ if(!d->error_retry)return(-7); switch(errno){ ...http://svn.xiph.org/trunk/cdparanoia/interface/cooked_interface.c
(please, don't drop me from the cc list!!)
it might be using the older sg interface, opening read/write to /dev/sgX char devices directly. In which case you can't test it with ide-cd, sadly.
Signature · 1 line
Jens Axboe
A message that is not in the archive (sent off-list or not yet received): 20060107112443.GA18749@section_eight.mops.rwth-aachen.de
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): 20060107115449.GB20748@section_eight.mops.rwth-aachen.de
A message that is not in the archive (sent off-list or not yet received): [email protected]
(please, don't drop me from the cc list!!)
Quoted 5 lines · Jens Axboe
(please, don't drop me from the cc list!!)
it might be using the older sg interface, opening read/write to
/dev/sgXchar devices directly. In which case you can't test it with ide-cd, sadly.
You wrote about accessing the drive with SG_IO while using ide-cd. So it is possible to use scsi commands though using ide-cd? I can't find any documentation on that, though. Could you point me towards it? I can try to adapt cdparanoia.
Cheers
Sebastian
Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
Quoted 5 lines · Sebastian
(please, don't drop me from the cc list!!)
Quoted 2 lines · Jens Axboe
(please, don't drop me from the cc list!!)
Quoted 8 lines
Quoted 3 lines
it might be using the older sg interface, opening read/write to
/dev/sgXchar devices directly. In which case you can't test it with ide-cd, sadly.You wrote about accessing the drive with
SG_IOwhile using ide-cd. So it is possible to use scsi commands though using ide-cd? I can't find any documentation on that, though. Could you point me towards it? I can try to adapt cdparanoia.
You can use SG_IO on any block device that accepts "SCSI" commands in the end, like ide-cd. Google for the sg driver documentation and you will find there are various ways to submit sg commands. cdparanoia likely uses the old variant of opening the char device and read/writing commands to it, if you convert it to SG_IO it could use that transport always (on 2.6 kernels and newer).
Signature · 1 line
Jens Axboe
(please, don't drop me from the cc list!!)
Yay, problem solved!
Quoted 11 lines · Jens Axboe
You can use
SG_IOon any block device that accepts "SCSI" commands in the end, like ide-cd. Google for the sg driver documentation and you will find there are various ways to submit sg commands. cdparanoia likely uses the old variant of opening the char device and read/writing commands to it, if you convert it toSG_IOit could use that transport always (on 2.6 kernels and newer).Signature · 1 line
Jens Axboe
Thanks guys for all your pointers. I googled for cdparanoia and SG_IO and found out that someone at Red Hat already patched cdparanoia to use SG_IO. http://people.redhat.com/pjones/cdparanoia/
If you're interested, the patches can be found in the src rpm:
http://people.redhat.com/pjones/cdparanoia/thomasvs/cdparanoia-alpha9.8-27.src.rpm
Well, I'm gonna submit an updated ebuild for Gentoo. Let's see how this thing evolves.
Thanks again for all your help, especially Jens!
Cheers
Sebastian
P.S.: I already reripped the test disc using ide-cd and the SG_IO-patched cdparanoia and the results are perfect. OMG, I bought Win XP Home 2 months ago because of this (so I can use Exact Audio Copy). I guess I can remove XP from my drive now and sell it to some wretched guy :) Harhar.
S.
Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
Quoted 6 lines · Sebastian
Yay, problem solved!
Quoted 7 lines
P.S.: I already reripped the test disc using ide-cd and the SG_IO-patched cdparanoia and the results are perfect. OMG, I bought Win XP Home 2 months ago because of this (so I can use Exact Audio Copy). I guess I can remove XP from my drive now and sell it to some wretched guy :) Harhar.S.
Yes, but now we need to find out why one interface fails while another works.. I have the same problem here using cdrdao when ripping entire disk images. I'd love to fix the real issue rather than work around it by having userspace use another interface. I would have thought that both interfaces should return the same data..
Brad
Signature · 3 lines
"Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so." -- Douglas Adams
(please, don't forget to me and [email protected])
Quoted 15 lines · Brad Campbell
Quoted 6 lines · Sebastian
Yay, problem solved!
Quoted 7 lines
P.S.: I already reripped the test disc using ide-cd and the SG_IO-patched cdparanoia and the results are perfect. OMG, I bought Win XP Home 2 months ago because of this (so I can use Exact Audio Copy). I guess I can remove XP from my drive now and sell it to some wretched guy :) Harhar.S.
Quoted 7 lines
Yes, but now we need to find out why one interface fails while another works.. I have the same problem here using cdrdao when ripping entire disk images. I'd love to fix the real issue rather than work around it by having userspace use another interface.
I would have thought that both interfaces should return the same data..Brad
I think cdrdao can use SG_IO if you tell it to. Check their documentation. Or did I misunderstand what you're saying?
Sebastian
Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
Quoted 1 line · Sebastian
(please, don't forget to me and [email protected])
The list convention regarding cc's is quite simple. ALWAYS use reply-to-all when responding to LKML mail. Therefore "please cc me, I'm not subscribed" is completely redundant.
Lee
Quoted 1 line · Sebastian
(please, don't forget to me and [email protected])
Did I? I'm sorry, I was *sure* I hit reply-to-all.
Quoted 6 lines
Quoted 4 lines
Yes, but now we need to find out why one interface fails while another works.. I have the same problem here using cdrdao when ripping entire disk images. I'd love to fix the real issue rather than work around it by having userspace use another interface. I would have thought that both interfaces should return the same data..
Brad
I think cdrdao can use
SG_IOif you tell it to. Check their documentation. Or did I misunderstand what you're saying?
Slightly.. If I'm not mistaken, we have a piece of software (for arguments sake let's call it cdparanoia). The stock software uses the ATA or ATAPI interface and produces bodged reads, the modified version you have uses SG_IO and produces accurate reads.
What I'm wondering is why the difference, and is there a problem with the ATA/ATAPI interface that leads to this that needs looking into.
*or* is it a userspace problem with the way cdparanoia is using that interface
*or* is it an inherent problem/limitation with that particular interface
I can reproduce this with cdrdao by reading a cd 10 times with all combinations of paranoia mode attempted. I get 10 x ~550 meg data.bin files that all differ.
I'm thinking I need to work up a script that diffs them by audio sector size chunks to see if there is a pattern there somewhere, but you would think that somewhere 2 reads in 10 would be identical.
Brad
Signature · 3 lines
"Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so." -- Douglas Adams
Quoted 13 lines · Brad Campbell
Quoted 1 line · Sebastian
(please, don't forget to me and [email protected])
Did I? I'm sorry, I was *sure* I hit reply-to-all.
Quoted 8 lines
Quoted 7 lines
Yes, but now we need to find out why one interface fails while another works.. I have the same problem here using cdrdao when ripping entire disk images. I'd love to fix the real issue rather than work around it by having userspace use another interface.
I would have thought that both interfaces should return the same data..Brad
Quoted 7 lines
Quoted 2 lines
I think cdrdao can use
SG_IOif you tell it to. Check their documentation. Or did I misunderstand what you're saying?Slightly.. If I'm not mistaken, we have a piece of software (for arguments sake let's call it cdparanoia). The stock software uses the ATA or ATAPI interface and produces bodged reads, the modified version you have uses
SG_IOand produces accurate reads.
It's the cdrom CDROMREADAUDIO vs SG_IO interface, there's not ata/atapi specific interface.
Quoted 2 lines
What I'm wondering is why the difference, and is there a problem with the ATA/ATAPI interface that leads to this that needs looking into.
Definitely, that's the big question. It could just be that cdparanoia enables extra error correction/checking, which it can do since it tailors the SCSI commands to its liking. Where as with CDROMREADAUDIO, you're basically stuck with whatever is coded into the cdrom layer.
Quoted 1 line
*or* is it an inherent problem/limitation with that particular interface
Could very well be. Once could use blktrace to see what the commands submitted to the drives look like and get some data out of that.
Sebastian, care to try one more thing? Patch your kernel with this little patch and try ripping a known "faulty" track again _not_ using SG_IO. See if that produces the same faulty results again, or if it actually works.
drivers/cdrom/cdrom.c +1 -1
@@ -426,7 +426,7 @@ int register_cdrom(struct cdrom_device_i cdi->exit = cdrom_mrw_exit;if (cdi->disk) - cdi->cdda_method =CDDA_BPC_FULL ; + cdi->cdda_method =CDDA_BPC_SINGLE ;else cdi->cdda_method =CDDA_OLD ;
Signature · 1 line
Jens Axboe
Quoted 22 lines · Jens Axboe
Sebastian, care to try one more thing? Patch your kernel with this little patch and try ripping a known "faulty" track again _not_ using
SG_IO. See if that produces the same faulty results again, or if it actually works.drivers/cdrom/cdrom.c +1
-1@@ -426,7 +426,7 @@ int register_cdrom(struct cdrom_device_i cdi->exit = cdrom_mrw_exit;if (cdi->disk) - cdi->cdda_method =CDDA_BPC_FULL ; + cdi->cdda_method =CDDA_BPC_SINGLE ;else cdi->cdda_method =CDDA_OLD ;Signature · 2 lines
-- Jens Axboe
Hi Jens,
I applied your patch, recompiled the kernel, rebooted and recompiled cdparanoia without the Red Hat patches. Extracting the first track of my test cd the result was the same as without the kernel patch with ide-cd using the cooked interface (md5 e8319ccc20d053557578b9ca3eb368dd).
Sorry :)
Sebastian
Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
Quoted 25 lines · Sebastian
Quoted 22 lines · Jens Axboe
Sebastian, care to try one more thing? Patch your kernel with this little patch and try ripping a known "faulty" track again _not_ using
SG_IO. See if that produces the same faulty results again, or if it actually works.drivers/cdrom/cdrom.c +1
-1@@ -426,7 +426,7 @@ int register_cdrom(struct cdrom_device_i cdi->exit = cdrom_mrw_exit;if (cdi->disk) - cdi->cdda_method =CDDA_BPC_FULL ; + cdi->cdda_method =CDDA_BPC_SINGLE ;else cdi->cdda_method =CDDA_OLD ;Signature · 2 lines
-- Jens AxboeHi Jens,
Quoted 6 lines
I applied your patch, recompiled the kernel, rebooted and recompiled cdparanoia without the Red Hat patches. Extracting the first track of my test cd the result was the same as without the kernel patch with ide-cd using the cooked interface (md5
e8319ccc20d053557578b9ca3eb368dd).Sorry :)
Well it's actually a good thing, because then at least it's not a bug with the multi-frame extraction. So my guess would still be at the error correction possibilities that the application has, in which case CDROMREADAUDIO is just an inferior interface for this sort of thing. It also doesn't give the issuer a chance to look at potential error reasons, since the extended status isn't available.
Signature · 1 line
Jens Axboe
Quoted 6 lines · Jens Axboe
Well it's actually a good thing, because then at least it's not a bug with the multi-frame extraction. So my guess would still be at the error correction possibilities that the application has, in which case CDROMREADAUDIO is just an inferior interface for this sort of thing. It also doesn't give the issuer a chance to look at potential error reasons, since the extended status isn't available.
For what it's worth; when allowing only flawless rips with cdparanoia (*) standard cdparanoia 9.8 and the SG_IO patched version of the same give me the exact same rips always, on my ATAPI Plextor "PlexWriter Premium", a drive that's specifically good at CDDA.
One thing that _is_ different is that the SG_IO version very frequently (soft) locks up the machine, with:
===
hdc: DMA timeout retry
hdc: timeout waiting for DMA
hdc: status error: status=0x7f { DriveReady DeviceFault SeekComplete DataRequest CorrectedError Index Error }
hdc: status error: error=0x7f { IllegalLengthIndication EndOfMedia AbortedCommand MediaChangeRequested LastFailedSense=0x07 }
ide: failed opcode was: unknown
hdc: drive not ready for command
hdc: ATAPI reset complete
hdc: irq timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: ATAPI reset complete
hdc: irq timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: status timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: drive not ready for command
hdc: status timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: DMA disabled
hdc: drive not ready for command
hdc: ATAPI reset complete
hdc: irq timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: ATAPI reset complete
hdc: irq timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: status timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: drive not ready for command
BUG: soft lockup detected on CPU#0!
Pid: 0, comm: swapper
EIP: 0060:[<c01d111f>] CPU: 0
EIP is at ide_inb+0x3/0x7
EFLAGS: 00000206 Not tainted (2.6.15-local-wc)
EAX: 00000180 EBX: c0329184 ECX: 2cb065ff EDX: 00000177
ESI: 00000282 EDI: 001403db EBP: c03290f0 DS: 007b ES: 007b
CR0: 8005003b CR2: b7c5d004 CR3: 1fd1f000 CR4: 000006d0
[<c01d16ca>] ide_wait_stat+0xa0/0xfd [<c01d07cb>] start_request+0x10c/0x1a7 [<c01d0b42>] ide_do_request+0x2bb/0x2f9 [<c01d00c5>] ide_atapi_error+0x53/0x78 [<c01da977>] cdrom_newpc_intr+0x0/0x2fd [<c01d0dfd>] ide_timer_expiry+0x195/0x1bf [<c0118ebe>] run_timer_softirq+0x140/0x181 [<c01d0c68>] ide_timer_expiry+0x0/0x1bf [<c0115b01>] __do_softirq+0x35/0x7d [<c0103c54>] do_softirq+0x38/0x3f ======================= [<c0115bda>] irq_exit+0x29/0x34 [<c0103b84>] do_IRQ+0x46/0x4e [<c01026ce>] common_interrupt+0x1a/0x20 [<c0100a73>] default_idle+0x2b/0x53 [<c0100af0>] cpu_idle+0x41/0x63 [<c02ea621>] start_kernel+0x13d/0x13f
hdc: status timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
hdc: drive not ready for command
hdc: status timeout: status=0x80 { Busy }
ide: failed opcode was: unknown
===
Ad infitum. Needless to say, I switched back to regular (unpatched from upstream) cdparanoia which doesn't show this problem. The patching I did was from this message earlier in this thread:
http://marc.theaimsgroup.com/?l=linux-kernel&m=113665002702563&w=2
First I tried with only the .sgio and .labels patches from that src.rpm applied, saw the problem, then retried with all of redhat's patches applied (just in case) but did not see an improvement.
As said, when things do work, the rips are exactly the same as the rips made with the standard cdparanoia.
(*) by "allowing only flawless rips" I mean using "cdparanoia -v -z -B" and only ever allowing spaces in the progressbar cdparanoia leaves behind as it goes along. When anything else (-, +, !, ...) appears, I re-rip the entire track, or the bit in which the error occured, until I have a "spaces only" copy. Two of such rips are always the same while on the other hand a non-spaces-only rip generally differs from any other rip made of that same track.
Rene.
Quoted 2 lines · Rene Herman
One thing that _is_ different is that the
SG_IOversion very frequently (soft) locks up the machine, with:
So you are saying that once you see the BUG: softlockup message the system is unresponsive and must be rebooted?
Lee
Quoted 6 lines · Lee Revell
Quoted 2 lines
One thing that _is_ different is that the
SG_IOversion very frequently (soft) locks up the machine, with:So you are saying that once you see the BUG: softlockup message the system is unresponsive and must be rebooted?
Well, a bit before that -- it takes a while for the softlockup timer to trigger. Yes and no, the machine's 'dead' while this is happening but in the end (minutes) it does recover in so far that it's no longer trying (but does leave around a cdparanoia in D state). Yes, I reboot, and then
things work again, until... well, they do not, which is quickly with
the SG_IO patched cdparanoia.
It's probably more useful to test with a very minimal CDDA extraction tool using SG_IO, if Jens has any such tool lying around, and if anyone _wants_ me to test. As said, I personally just switched back to using regular cdparanoia, which is working fine for me.
Rene.
You can find the ebuild at
https://bugs.gentoo.org/show_bug.cgi?id=118189 in case you're interested :)
Sebastian
Signature · 1 line
"When the going gets weird, the weird turn pro." (HST)
Quoted 4 lines · Sebastian
You wrote about accessing the drive with
SG_IOwhile using ide-cd. So it is possible to use scsi commands though using ide-cd? I can't find any documentation on that, though. Could you point me towards it? I can try to adapt cdparanoia.
In 2.6 yes. The SG_IO ioctl works on any block device fd. The commands you can issue then depend upon the mode of opening. Some commands require root, generally safe ones read or write access.
Due to bugs still present in the block layer its also the only way to reliably copy a data CD as well.
Alan