Computer Vision on a brain

A new brain-computer-interface technology could turn our brains into automatic image-identifying machines that operate faster than human consciousness.

Researchers at Columbia University are combining the processing power of the human brain with computer vision to develop a novel device that will allow people to search through images ten times faster than they can on their own.

Darpa, or the Defense Advanced Research Projects Agency, is funding research into the system with hopes of making federal agents' jobs easier. The technology would allow hours of footage to be very quickly processed, so security officers could identify terrorists or other criminals caught on surveillance video much more efficiently.

The "cortically coupled computer vision system," known as C3 Vision, is the brainchild of professor Paul Sajda, director of the Laboratory for Intelligent Imaging and Neural Computing at Columbia University. He received a one-year, $758,000 grant from Darpa for the project in late 2005.

The system harnesses the brain's well-known ability to recognize an image much faster than the person can identify it.

"Our human visual system is the ultimate visual processor," says Sajda. "We are just trying to couple that with computer vision techniques to make searching through large volumes of imagery more efficient."

The brain emits a signal as soon as it sees something interesting, and that "aha" signal can be detected by an electroencephalogram, or EEG cap. While users sift through streaming images or video footage, the technology tags the images that elicit a signal, and ranks them in order of the strength of the neural signatures. Afterwards, the user can examine only the information that their brains identified as important, instead of wading through thousands of images.

No existing computer vision systems connect with the human brain, and computers on their own don't do well at identifying unusual events or specific targets.

"The major weakness of computer vision systems today is their narrow range of purpose," says Steven Gordon, an information systems and technology professor at Babson College in Massachusetts. "You cannot take a system that is intended to recognize faces and apply it to recognizing handwriting or identifying whether one object in a photo is behind another. Unlike a computer, which can perform a variety of tasks, a computer vision system is highly customized to the task it is intended to perform. They are limited in their ability to recognize suspicious activities or events."

People, on the other hand, excel at spotting them. The new system's advantage lies in combining the strengths of traditional computer vision with human cortical vision.

For example, when a computer searches for vehicles, it will identify and discard parts of the image that contain water. The human user, who is more likely to easily spot oddities, can then look only at the parts of the image that matter. This could allow time-sensitive searches to be performed in real time.

Gordon believes that the technology would be most appropriate for analyzing stored video and for intelligence gathering.

"Conceivably, the proposed solution could be applied in quasi-real-time to allow a single human to monitor ten times as many sites as he or she would otherwise monitor," says Gordon.

The Columbia team is currently working on making the system more robust and reducing instances of false positives. They plan to demonstrate the technology for Darpa in a few months.

Source: http://www.wired.com/medtech/health/news/2006/07/71364