Scientists display what the brain "sees"

[Kittamaru]

http://gizmodo.com/5843117/scientist...brain-activity

UC Berkeley scientists have developed a system to capture visual activity in human brains and reconstruct it as digital video clips. Eventually, this process will allow you to record and reconstruct your own dreams on a computer screen.

I just can't believe this is happening for real, but according to Professor Jack Gallant—UC Berkeley neuroscientist and coauthor of the research published today in the journal Current Biology—"this is a major leap toward reconstructing internal imagery. We are opening a window into the movies in our minds."

Indeed, it's mindblowing. I'm simultaneously excited and terrified. This is how it works:

They used three different subjects for the experiments—incidentally, they were part of the research team because it requires being inside a functional Magnetic Resonance Imaging system for hours at a time. The subjects were exposed to two different groups of Hollywood movie trailers as the fMRI system recorded the brain's blood flow through their brains' visual cortex.

The readings were fed into a computer program in which they were divided into three-dimensional pixels units called voxels (volumetric pixels). This process effectively decodes the brain signals generated by moving pictures, connecting the shape and motion information from the movies to specific brain actions. As the sessions progressed, the computer learned more and more about how the visual activity presented on the screen corresponded to the brain activity.
An 18-million-second picture palette

After recording this information, another group of clips was used to reconstruct the videos shown to the subjects. The computer analyzed 18 million seconds of random YouTube video, building a database of potential brain activity for each clip. From all these videos, the software picked the one hundred clips that caused a brain activity more similar to the ones the subject watched, combining them into one final movie. Although the resulting video is low resolution and blurry, it clearly matched the actual clips watched by the subjects.

Think about those 18 million seconds of random videos as a painter's color palette. A painter sees a red rose in real life and tries to reproduce the color using the different kinds of reds available in his palette, combining them to match what he's seeing. The software is the painter and the 18 million seconds of random video is its color palette. It analyzes how the brain reacts to certain stimuli, compares it to the brain reactions to the 18-million-second palette, and picks what more closely matches those brain reactions. Then it combines the clips into a new one that duplicates what the subject was seeing. Notice that the 18 million seconds of motion video are not what the subject is seeing. They are random bits used just to compose the brain image.

Given a big enough database of video material and enough computing power, the system would be able to re-create any images in your brain.

In this other video you can see how this process worked in the three experimental targets. On the top left square you can see the movie the subjects were watching while they were in the fMRI machine. Right below you can see the movie "extracted" from their brain activity. It shows that this technique gives consistent results independent of what's being watched—or who's watching. The three lines of clips next to the left column show the random movies that the computer program used to reconstruct the visual information.

Right now, the resulting quality is not good, but the potential is enormous. Lead research author—and one of the lab test bunnies—Shinji Nishimoto thinks this is the first step to tap directly into what our brain sees and imagines:

Our natural visual experience is like watching a movie. In order for this technology to have wide applicability, we must understand how the brain processes these dynamic visual experiences.

The brain recorders of the future

Imagine that. Capturing your visual memories, your dreams, the wild ramblings of your imagination into a video that you and others can watch with your own eyes.

This is the first time in history that we have been able to decode brain activity and reconstruct motion pictures in a computer screen. The path that this research opens boggles the mind. It reminds me of Brainstorm, the cult movie in which a group of scientists lead by Christopher Walken develops a machine capable of recording the five senses of a human being and then play them back into the brain itself.

This new development brings us closer to that goal which, I have no doubt, will happen at one point. Given the exponential increase in computing power and our understanding of human biology, I think this will arrive sooner than most mortals expect. Perhaps one day you would be able to go to sleep wearing a flexible band labeled Sony Dreamcam around your skull. [UC Berkeley]

[leopold]

what is going to happen to the creation of our genius?
http://www.sciforums.com/showthread.php?t=109699
even our worst nightmares can actually become a reality.

mind boggling to say the least.

[Stryder]

It's funny how sometimes today's "New" is actually yesterday's "Old".

I'm pretty sure, in fact I am absolutely positive there are already more advanced systems out there, however due to all that cold war bollocks and the superspy mythology of old, it's all been hidden behind the cloak. In fact it's no longer "cloak and dagger" but more "Cloak and scalpel".

[Kittamaru]

It's funny how sometimes today's "New" is actually yesterday's "Old".

I'm pretty sure, in fact I am absolutely positive there are already more advanced systems out there, however due to all that cold war bollocks and the superspy mythology of old, it's all been hidden behind the cloak. In fact it's no longer "cloak and dagger" but more "Cloak and scalpel".

It wouldn't surprise me to be honest

[wynn]

http://gizmodo.com/5843117/scientist...brain-activity

And what would be the use of such a "brain recorder"?

Other than entertainment?

Imagine that. Capturing your visual memories, your dreams, the wild ramblings of your imagination into a video that you and others can watch with your own eyes.

Yes, in earlier times, people went through the trouble of learning art and crafts so that they could express themselves. Silly them!

[Kittamaru]

Signal - it isn't so much about the capability of showing what the brain sees, but being able to understand how it works - think about it, this could greatly enhance the technology of, say, prosthetic eyes... imagine the implications for someone who goes blind due to an accident... now, it's sorta "oh, well, sucks to be you"... this could be used to, literaly, wire a hi-def camera into the brain

[wynn]

And who could afford such prosthetic devices?

[Kittamaru]

And who could afford such prosthetic devices?

How long ago was it that people said the same thing about hearing aids and cochlear implants

[wynn]

The technology required for hearing aids and such is vastly different than for a "brain recorder".

[river-wind]

And what would be the use of such a "brain recorder"?

Other than entertainment?

1) Recording dreams (useful for insight into personality and also into diagnosing mental disorders)
2) Communicating with those unable to speak or move. Identifying coherent function in semi-catatonic patients.
3) Massive step towards inspecting the function of the brain directly. Currently, we have to rely on the brain to translate whats in the brain into language, or we look are very rough blood-flow mappings like fMRI. Being able to actuall translate brain-coded information is HUGE.

[leopold]

And who could afford such prosthetic devices?

who could afford the 60 megabyte 5 pound hard drive of the 60s?
3 or 4 grand? are you kidding?
who could afford the eniac?
a roomfull of equipment that required a monster amount of power.
i'm afraid to even guess at the price tag.

both can be had today, brand new, for 400 dollars.
get one at a yard sale for a hundred or less.

[Stryder]

Incidentally, if it's rigged up en-mass it would likely use matricing of already abundant telephony arrays and cloud computing, which would reduces costs dramatically.

[Dr Mabuse]

Wow so 'Brainstorm' really was a futuristic movie huh?

[Telemachus Rex]

And what would be the use of such a "brain recorder"?

Other than entertainment?

Interrogation comes to mind. Imagine what a killer sees in his minds eye when being questioned about a murder.

[CptBork]

How can anyone doubt the enormous potential applications if this research pans out? Are you kidding me? If you can probe someone's visions, imagine how easy it would be to interrogate someone by practically sucking the information right out of their brain? No need for torture under any circumstances whatsoever, just a massive invasion of privacy in cases of emergency.

I've been thinking how great it would be if computers were intelligent enough to construct programs and works of art based on a user's suggestions and corrections, for those who simply don't have the talent or knowledge to do it all themselves. You say "draw me a purple dragon!" and out pops a purple dragon, then you say "make the wings spikier" and the computer makes it so- I always thought that would make for an unprecedented cultural and artistic revolution. So I figure most of us are pretty decent at creating mental imagery- even in the absence of intelligent, interpretive machines, we could still create our own movies, music and artwork just by imagining it and then perhaps using our imaginations again just to correct all the fine details. Mental Photoshop and all that. Heck, imagine "telepathic" cellphone service for a mere extra $20/month?

[Asguard]

not to mention how much insite could you get with regard to halucinations by use of something like this? Even if not directly used it could show you where the aberations are alowing for future developments potentually targeting those cells directly

[leopold]

You say "draw me a purple dragon!" and out pops a purple dragon, then you say "make the wings spikier" and the computer makes it so-

or how about the actual living, breathing, thing?

the food replicators of "star trek" might not be far off.

the linking of artificial intelligence into the human brain will have undreamed of consequences.

[Michael]

It's funny how sometimes today's "New" is actually yesterday's "Old".

I'm pretty sure, in fact I am absolutely positive there are already more advanced systems out there, however due to all that cold war bollocks and the superspy mythology of old, it's all been hidden behind the cloak. In fact it's no longer "cloak and dagger" but more "Cloak and scalpel".

Sounds like you have the making's of a best seller, if you have the real info.

[Michael]

The technology required for hearing aids and such is vastly different than for a "brain recorder".

It is true the fMRI are so expensive many major cities only have one up to a few (even none) BUT who knows what the future may hold? Machining making small fMRIs much like personal laptops evolved from room sized mainframes.

[Michael]

How can anyone doubt the enormous potential applications if this research pans out? Are you kidding me? If you can probe someone's visions, imagine how easy it would be to interrogate someone by practically sucking the information right out of their brain? No need for torture under any circumstances whatsoever, just a massive invasion of privacy in cases of emergency.

I've been thinking how great it would be if computers were intelligent enough to construct programs and works of art based on a user's suggestions and corrections, for those who simply don't have the talent or knowledge to do it all themselves. You say "draw me a purple dragon!" and out pops a purple dragon, then you say "make the wings spikier" and the computer makes it so- I always thought that would make for an unprecedented cultural and artistic revolution. So I figure most of us are pretty decent at creating mental imagery- even in the absence of intelligent, interpretive machines, we could still create our own movies, music and artwork just by imagining it and then perhaps using our imaginations again just to correct all the fine details. Mental Photoshop and all that. Heck, imagine "telepathic" cellphone service for a mere extra $20/month?

Those are some awesome ideas. I wonder if the perfect Assassin would be a Buddhist Zen Master, ha, but then they'd have no interest in assassinating anyone