Hi all, my first post here and my first question as well.

Does any one know of or know how to convert a video picture signal in to audible music?

Say we take the frequency range of the visable light and convert it without embelishment or deliberate form to the frequency range of audible sound.

Has this been done before do you know?

I am just curious to listen to what "a storm at sea" video for example would sound like (sight to sound)

The final audio file would include actual audio from the video I would suspect as well or this can be determined as you want.

I've heard of music being generated from fractals. I think you would just get off the scale noice.

Conceptually the idea behind the question is that if we can generate a true audio representation of the visual then a blind person could wear a special video camera and listen to what he would see with it.

The ideal being that with training he could mentally picture the light or vista by using the sound generated.

The only difficulty I think would be the ability to show depth of field although different intensities and textures might help.

Possibly combined with some sort of ultrasonic sonar would be used.

( I think this is already in use but I am not sure)

I apologize for thinking that this idea would be 'just for fun' (right after I saw the title I thought that way). But such help to the blind is a great idea, I'll say.

Quantum,

You would have to create your mapping. You would have to for instance, equate certain greyscales to a frequency range....

Perhaps you could try to write it for an art gallary. Your program could detect the mood of the picture and then could then play different music.

hey guys this is great....any other ideas??

The biggest problem I think would be translating the 3 dimensionality.

But this would possibly be a later refinement.

Does any one know of software that can possibly start the process of translating visual to sound.

Sound not necessarilly conventional music ( chromatics etc)

You really need one major software: Matlab.

Why do you think of Matlab? I had a look at a couple of websites and it seems to be pure mathematics.

As i am no mathematician maybe you can tell me why it would be a good starter?

Matlab:http://www.mathworks.com/

Matlab is by far the best, general tool for all technical modelling.
Depending on how you want to map an image to a frequency range (pixel or block), you will have to model it using a tool that allows this modelling... Have you used matlab before? Also it is not pure mathematics, you have some progamming to do. Also, matlab = MATH lab.

No I have never used Matlab and to tell the truth if I have to learn the math and programing the whole project is in a dither.

I am currently talking to two software design houses to get their opinions and so far I have had nothing to get excited about.

I am a little surprised that this hasn't been done in this way before.... I was hoping that there was software already available to do the conversions ( directly with out a lot of mapping) but it seems this is not the case.

There are options like just creating a 3 dimensional positioning system using two cameras, one for each ear so that a blind person could tell exactly where he was in a room of 4 or more walls. Or distinguish between horizontal and vertical surfaces making moving out side a little more easy.

By using reverberation as a clue to depth of field ( distances) etc

Well I think a simple mapping in 2D could just take the pixel's frequency, and then do some mapping depending on the application into a digital audio. You could then use any of the tools out there that map digital to analog. You might also try using sonar like bats do.

It's easy to create a sound representation of a picture, but there's no guarantee that the sounds would qualify as 'music.' If you think about it, this is what you do any time you send an image file over a modem.

I would think that to use a chromatic or tone representation would hinder the project.The noise or sound generated would have to reflect the infinite gradients of colour and position.
The other thing to consider is that whne we look we tend to focus on a given point in 3 dimensional space, all other objects are some what out of focus. This also would complicate the way the cameras where untilised and sound was extrapulated.

the main thing here is the idea of "mapping" above. there is no direct way to translate sight to sound, because sight is EM radiation, and the other is compression waves in a medium. you can map one to the other, but you have to invent a translation method, injecting your own personal tastes along the way. For instance, let's say you decide that Middle C on a scale should equate to mid-tone grey. You've already introduced cultural bias (why is that C "middle" C?), and you then have to somehow add colors to the mix.

This could be avery cool project, but the first step would be to define how the light waves would be translated into sound waves.

Actually I think it is a great challenge in dimensional thought.

For instance when you take an entire video signal and convert to sound. Reds would be a low pitch sound, greens would be a middle pitch sounds and blues would be a higher pitch sounds.

The thing is that in a straight conversion what we would actually hear is only the percetage of colour to a percentage of sound.
For example if 60% of the vista is redish then 60% of the sound heard may be from two octaves below Middle C and one octave below middle C.

Say we have a vista the consists of 20% red, 60% green and 20 % blue.

The problem is that light vista is percieved with location where as sound location is harder to achieve. The vista being "linea"
and the sound being "Alinea" (for want of a better word)

I thought OK we have stereo headphones that each side consists of three speakers each but I think that even if we position the sound around the ear, location is very hard to distinguish.

The only real way of distiguishing location seems only to be achieved by using stereo effects.

I just contacted our Australian institute for the blind and they have given me the following link which I found to be very interesting.
[URL=http://www.seeingwithsound.com/voice.htm]

I have run the demo they offer and unfortunately I find their approach confusing. In that they have created too much information I feel.

What do you think?

oops! First time I use urls here

http://www.seeingwithsound.com/voice.htm