Why can't sounds be edited like colours?

Say in Photoshop I want to remove all of the red from an image, it is easy. Say I want to remove someone from the background of a photo it is easy. But when it comes to music packages, they are extremely limited...

1/ Say I want to remove a piano from a rock track.
2/ Say I want to change all of the individual sounds into individual colours so that I can just edit the colours in a paint package, and put them back into sounds later.

I think I am sort of concluding that our sound editing should be as simple as our image editing, but it isn't. And it seems to have some sort of scientific reasoning that I haven't figured out.

And another thing... even after all of these years with computers we have computers that can draw, and still no vocals built into the computer. Why is it so hard to replicate sounds?

If you take the sonar of a bat, and somehow reproduce it as images maybe you could edit sounds more easily?

 

Log in or Sign up to hide all adverts.

If you have the master recording on tape or whatever you can remove anything that was recorded onto it very easily in a recording studio. There are "tracks" onto which each instrument is recorded which can be removed when re-recording the piece of music.

 

Log in or Sign up to hide all adverts.

Yes just about all sound and video editing programs allow you to do that. Also, if you've watched any of the CSI or forensic TV shows, I'm betting you can remember times when they had to remove various levels of background noise or sound to make it easier to understand the sounds they are interested in?

But in a way there is a question as to why sound editing by the average person is not as common as photo editing? My guess is that video and music editing is considered something for experts and not the average person. There is a perceived high learning curve that most people don't want be involved with.

 

Log in or Sign up to hide all adverts.

You really need the master recording to edit it in order to remove each instrument from each track for without it you don't have that ability and therefore can't edit it perfectly. The master recording is the key and few people are privileged enough to have that.

 

Well I use both as a computer games designer, and sound editing is way behind image editing. When you look at a sound sample it is all intermixed, and taking out a piano is like taking white out of white noise. Maybe it is just sound sampling that needs to change?

 

With color editing we are often dealing with single colors (more or less), but audio tends to use many sounds in groupings, such that the a piano sound is not just one wavelength. Sound also has other features like attack, sustain and release which defines the profile of the sound in terms of loudness, for example. In color that would be analogous to the intensity of a colored streak as a function of position, which may be hard to isolate in whole due to the thinnest color causing blending.

As cosmic traveler said, if you had the master recording with multi-tracks this is analogous to layers in photoshop. This offers the ability to isolate layers and tweak each instrument.

 

I don't count layers in this. Just a single Photoshop Photograph, v's a sound sample. You are then allowed to create layers from these two single data banks. So in the Photo I can remove a person, but in the sound bank I cannot remove the piano.

 

Correct, unless you have the master recording, then you could.

 

There is a fundamental difference between sound and imagery that makes this impossible to treat the way you are thinking of it.

You are probably aware that sound is stored as a waveform (a pair for stereo). This waveform was generated by mixing all the waveforms of all the sounds that are combined into the track. When waveforms are mixed, the amplitude of each sound wave is added to form the amplitude of the final mix, time sample by time sample.

Once you have added numbers together, there is no way to split them apart again, because that information has been lost.

Unless, as Cosmic mentioned, you have all the original tracks, as in the old days when they kept multitrack masters. And of course if you made the mix yourself, you can do the same thing. You probably already figured out that you'd better keep all your track changes step by step, in case you make a mistake or change your mind and want to go back to an earlier step.

There are some tricks you can play with signal processing to sort-of remove certain sounds. You will notice a lot of audio applications include filters to remove vinyl record noises, like pop and skip, and some apps have voice removal capability. Just by filtering you can sometimes come close to removing the sound.

Also these apps give you a way to edit the waveform sample by sample. But without any idea how to remove the sound, you will get nowhere.

You didn't have this problem with images because there is no information lost in an image, unless it is formed from overlays, but that's different. Most images don't involve the successive numerical addition of pixels like you have with successive numerical addition of samples.

One more thing: if the sound is repeatable, such as a pure tone, that is, something you can generate yourself, then it's possible to take the generated sound and subtract it from the mix and erase just that sound. But even this can be difficult to do with out knowing in advance the amplitude and phase of the sound you are trying to remove.

 

As has been said previously, if you have access to the master recording, you can then manipulate any discrete master track in innumerable ways.

As an example. Flo Rida was granted permission to "sample" vocal elements from the 1962 song "Something’s Got a Hold On Me" by the late Etta James. Using modern digital mixing/effects technology, this borrowed sample was then seamlessly woven into an entirely different song which has climbed to #1 on the urban/contemporary charts...

Flo Rida - Good Feeling

 

Do you think that stereo sound sampling could be stored in a way that didn't combine numbers together? Maybe there is a flaw in that approach to begin with. Maybe directional sound could be more directional? Say I am recording an orchestra, maybe the direction of the sounds could be used to break the sounds up into parts. Here I am comparing a Camera to a Digital Recorder. But a person in a photo still can be removed if they are a ghost, which is the same as combining numbers together... in fact it is easier. I suppose that the equivalent photo is to have everything in the photo ghosted, and overlapping. I'm wondering if you can remove the ghost from sounds.

Think of it this way.. as a human I can tell the piano from the drum. So I am doing some sort of editing that computers aren't doing. Probably a Neural Network could do it.

 

An orchestra would probably be recorded by placing two microphones in front of the group to capture the entire performance. By the time the signal from the oboe (or whatever) reaches the microphones, it is well blended into all the other sounds of the group. Not to mention that it is no longer a single wave. It bounces off the walls, floors and ceilings thereby hitting the two microphones many times from many different angles.

It's the many different signals arriving at our ears at different times and phases that allows us to localize where a sound came from. It also allows us to determine the size of the space the performance was recorded in. (This maybe enhanced artificially by means of reverb and/or other processing techniques.)

 

You would think then that the brain needs far more resources available for de-mixing sounds than images.

 

Hearing is much more complicated than most people realize. Here's an excellent article geared towards the recording engineer, but I think most people can get some benefit from reading it.

http://www.moultonlabs.com/more/microphone_vs_the_ear/P0/

 

Thanks! Anyway I would love to see somebody write a Neural Network attempt at splitting sounds from sound samples.

 

The imagination of a sitcom screenplay writer can capture (from memory) what constitutes the voices of various actors and have them engaging in internal, fictional conversations -- so as to test out parts of an episode's story before typing it out. With practice, I suppose one could also eventually edit out a particular instrument from the commercial music songs we sometimes "play in our heads" from memory.

Accordingly, a computer program capable of doing similar would seem to require what amounted to a specialized artificial intelligence augmenting it (i.e. utilizing far more rules of analysis and featuring a literal background understanding of music and sound equivalent to not only the experiential and intuitive aspects that a human is familiar with, but also the complexity of underlying non-conscious processing transpiring in the brain that we are normally unaware of). So as to then be able to distinguish the defining signature of one instrument from another, and then isolate and keep track of its distributed "components" through the whole range of maze-entangled frequencies over time, so as to go about either removing or modifying it.

Needless to say, an editing program enhanced in this manner would require both some hefty RAM when it was active and storage space for itself on the harddrive, even if such was invented.

Since the computerized process of colorizing of B&W movies "might" resemble the following analogy in certain respects, I'd probably be understating it significantly if I said that the above approach to sound would probably be like keeping tabs on only the green-colored needles in a constantly re-arranging haystack of diversely colored needles over time (or rather, through all the static "nows" constituting that patterned sequence of changes), with the green-colored needles themselves having their own degrees of variation to further confuse matters.

 

An interesting experiment is to take a stereo wave file. Split the left and right into two separate tracks. Now reverse the phase of one track, then recombine them into a mono signal. Everything which was in the center of the stereo image will have cancelled itself out, thereby removing it.

 

Just concentrating on music, and no other sounds apart from instruments. Let's take the piano, there are maybe 100 piano sounds on a synthesiser with say 1000 total sound banks. If I close my eyes and let somebody randomly choose any sound I can identify all 100 piano sounds from 1000 total sounds. I will also think that most of them sound the same. I will also say if a piano is electric or Grand.

My idea is that you give a Neural network 100 piano sounds to work with. Now, can it remove any piano sound from a sound sample of an orchestra?

If it can, then you don't really need a huge Hard Drive for 100 pianos, 100 drums, 100 trumpets, 100 trombones. You might even get the Neural Network to work with just 10 of each sound. Once the Neural Network is set up you can delete the sounds anyway, it will have stored a virtual piano, a virtual drum etc in a sort of maze of patterns.

 

Why? When you already have the master recording you don't need to create something like that at all.

 

Say I wanted to use a drum track from a Deep Purple album, or take a sample of Michael Jackson from Thriller. But synthesisers also allow you to put a sound into each key on the keyboard, so a Deep Purple synth could be snatched into your keyboard. Or say you had a guitar track in a Genesis album, and you wanted to change it into a piano track.