Multimedia compression

Basically, there's two interrelated research agendas, both of which need to make it out of the lab for this sort of compression to make it to the real world:

[1] Doing the decomposition. The three major ways people are thinking about solving this problem right now are

[A] The mathematical approach -- in a nutshell "Let's separate out N signals from one, under the assumption that the best separation makes each separated signal carry unique information. The best starting point for understanding how these folks think about the problem is:

http://www.cnl.salk.edu/~tony/ica.html

While many of these papers think in terms of "N microphones to listen to a performance of N instrumentalists" as the starting signal, this is just to make the math easier -- in practice the ideas can be extended to the more general case.

[B] Auditory scene analysis -- in short, take the analogy from computer vision, where raw camera (or retina) output gets broken up into different "maps" coding motion, color, shape, ect, and apply it to audition. These maps should (in theory) make the process of doing separation a lot easier. The good initial resource for this approach is:

http://sound.media.mit.edu/~dpwe/AUDITORY/

[C] Get access to the 24-track master tapes (uh, I think I just dated myself :-), and avoid the whole decomposition problem. Technically easy, of course, but might not be practical.

[2] Once you've done the decomposition, create encoders specialized for specific types of sounds. For certain types of specialized sounds, this field is very mature (i.e. spoken speech). For other types of sounds, though, you're basically going to have to

[1] Look at the best music synthesis algorithms for the sound type, and see how easily the algorithm can be "inverted".

[2] Hope a speech codec works well on the sound (there are a few codecs specialized for singing voice that take this approach ...).

[3] Do original research on the problem.

One big problem with a field like this is that its hard to get people motivated to work on the "little problems" described above, if its unclear how success on the little problem is going to solve any big problem. The hope with Structured Audio is to solve this "meta-problem" -- by having a fielded platform out there, ready to support new approaches to sound encoding in pilot applications, SA will help motivate research in the whole area.

In particular, the politics of MPEG is very different than the politics of IETF -- "joining the process" of creating a new coding standard is a deep committement of time and resources and money in MPEG-land (note that I'm not an MPEG member, so I'm outside the process, largely because of the committment it takes to meaningly participate). But implementing a new codec _in_ Structured Audio is a whole different ballgame -- you could get a team together under the rubric of the IETF, or you could just take the traditional Free Software approach and start a grass roots effort like LAD. You don't need anyone's permission to write a SAOL program ...

									--jl

-------------------------------------------------------------------------
John Lazzaro -- Research Specialist -- CS Division -- EECS -- UC Berkeley
lazzaro [at] cs [dot] berkeley [dot] edu www.cs.berkeley.edu/~lazzaro
-------------------------------------------------------------------------

How do you know it? Where I could read tests about it?
I have used layer 3 with 320 kbps for compressing radio and cassette recordings (both music and sound samples snipped from them), so, most probably I have used the right compressor. But is layer 3 with 320 kbps somehow not enough for studio mikes (voice, guitar, etc.) or for digital synth tracks?

Of course, for compressing the final mix (20-bit or greater) of studio work, the 16-bit MP3 is not enough. But 16-bit could be enough for individual sounds, vocals, etc. However, I would not use any MP3 compression for even those because I would record only with 20-bit A/Ds in studio. I have used MP3 only because I have nothing better than 16-bit A/Ds. So, what compressor I could use for 20- or 24-bit material? Should we look at ourself for 20- or 24-bit compressor or can MP3 or AAC handle them (similarly than JPEG can handle 16-bit/channel images)?

Note that 16-bit is most probably enough for mic recordings, but the extra bits are needed for having extra room. I would not want to dither 20-bit recording down to 16-bit even it would fit very well.

Juhana

I think Philips MPEGLIB can encode 24bit files, their MP2 software encoder is the same used in DMX SAT radio encoders , and other DSP platform.

I say I would prefer MP2 over MP3, because MP3 *WAS DESIGNED* for lower bitrates. ( 128-192kbit).

Plus consider the fact that MP2 has better random-seek properties (no long frame dependencies), not to mention higher speed when encoding/decoding material.

If you ask me, if I were designing a system with the best possible audio quality while achieving 1:4 compression I would chose MP2 or alternatively APT-X ( http://www.aptx.com/ )
Unfortunately APT-X is not available as a software-only CODEC, it is used in some cinema audio systems (DTS) , and doesn't use perceptual coding, and has an amazing 2.9ms codec delay at 44.1kHz.
The granularity of editable regions is 80usecs = 2 samples, that means you can edit an APT-X file with almost the same precision as a PCM file. ATPX is some kind of ADPCM but which encodes subbands with different precision to reflect the frequency resolution of the ear.

But I guess you are looking at a totally free codec :-) therefore there is only MP3 , ISO MP2 encoder and perhaps a good VORBIS (the patent-free audio codec which matches MP3 quality)

Benno.

I will second Paco's synopsis of Shorten.

I have close to 100 concerts that were compressed using the lossless setting in Shorten, and they are *exact* copies of the original when decompressed. You can do an md5sum of the wav file prior to compressing it and it will match after it is uncompressed later on.

I can typically cut a wave file size in half and it is lossless...I don't think you
can ask for more then that...at least not yet anyway. I have not tried to do lossy compression with Shorten so I cannot comment on it's performance in the lossy realm...

Another thing is that WinShorten *will* stream shorten files so that you do not have to convert them to wav to listen to them. I setup a machine to do that very thing so that I didn't have to burn my extensive SHN library to audio CDR or wav file prior to listening. I haven't found a streaming version of Shorten for linux, so I grabbed the source code the other day and am looking into what it would take to do it...shouldn't be to difficult.

~Db

----- Original Message -----
From: <[email protected]>
To: Benno Senoner <[email protected]>
Cc: Sergey <[email protected]>; <[email protected]>; <[email protected]>; John Lazzaro <[email protected]>
Sent: Tuesday, June 20, 2000 12:08 PM
Subject: [linux-audio-dev] Re: Multimedia compression

Hey... this is fairly new. Somebody hacked XMMS so that it plays SHN files directly. The relevant info is in the text below.

Note that you can also do this:

[email protected]%> shorten -x <file>.shn - | play -t wav -

from the command line to play a SHN without making the intermediate .WAV file. It works fine, but you don't get a nice GUI or the ability to pause, ffwd, or rewind the audio.

I ahven't checked out the hacked XMMS yet, but it's the talk of the town on www.etree.org.

list.

minor

Just in case anybody out there isn't familiar with perl, I though I'd send this email out to help them. Attached is my script for playing SHNs out of directories or off of a CD.

To use it, put it in your path somewhere (make sure it's executable!), and type:

shnplay /my/shn/file/directory/or/CDROM/path

That's is. It will play each file with a ".shn" extention in that directory without making big WAV files as an intermediate step.

SInce the shorten algorithm is so fast, playing SHN files like this only uses up about 10% of my CPU. (550MHz Athlon.)

BTW: it's nice to have a topic on this mailing list aside from "how do I get this card working" or "is there planned support for this card" and the like. Those are all good posts to this list, but we never really have too many actual discussions about stuff. We just help each other make the boxes make noise.

Whoops... forgot the attachment....

sorry,
--Paco

TEXT/PLAIN attachment: perl script for playing SHN files