More on DeepMind: AI Startup to Work Directly With Google's Search Team

Google has been buying a lot of crazy stuff lately. At least eight robot companies, including humanoid robot-maker Boston Dynamics. Nest, the smart-home company that designs thermostats and smoke detectors. And now DeepMind, an artificial intelligence startup.

Taken together, the deals might all seem to add up to Skynet. But sources said DeepMind is actually being inserted into Google’s oldest team: Search.

Or, as search is known at Google today, the “Knowledge” group — so-called because it no longer just finds keywords on Web pages, but instead connects larger concepts. Knowledge is led by Google SVP Alan Eustace, but DeepMind will work closely with a team led by Jeff Dean, a near-15-year Google veteran best known for his work on distributed systems.

By contrast, the pack of acquired robots will report to former Android boss Andy Rubin, and Nest will continue to be managed by Tony Fadell (who is to report directly to Google CEO Larry Page).

“Manhattan Project for AI”

It may be hard to believe given the $400 million price tag (perhaps more, with earn-outs), but DeepMind is seen by Google as primarily a talent acquisition.

London-based DeepMind had not yet released any products, but sources said it was working on at least three: “A game with very advanced game AI, a smarter recommendation system for online commerce and something to do with images,” is how one source described it.

DeepMind employs a team of at least 50 people and has secured more than $50 million in funding, and it competed for talent with companies like Google, Facebook, Baidu, IBM, Microsoft and Qualcomm. As far as AI goes, it was perhaps the only startup name that could be included on that list.

“If anyone builds something remotely resembling artificial general intelligence, this will be the team,” one early investor in DeepMind told Re/code today. “Think Manhattan Project for AI.”

DeepMind’s technology does become part of Google through the acquisition. There are traces of it around the Web, including three U.S. patent applications around reverse and composite image search and a paper about how a set of algorithms can learn to play and beat expert human players of the Atari games Breakout, Enduro and Pong.

Facebook had also been interested in the team at DeepMind, as The Information reported, though a source described talks late last year with Facebook CEO Mark Zuckerberg as aimed more at scooping up some of the deep learning researchers than buying the full company.

Amir Efrati at The Information anticipated the Google acquisition in a paywalled article in December, which laid out the “arms race” in deep learning between tech giants that are increasingly eager to hire researchers in the small field as part of general efforts to make their products smarter. Efrati also reported that the DeepMind Atari demonstration had impressed conference attendees in December.

Going Deep

Deep learning is a form of machine learning in which researchers attempt to train computer algorithms to spot meaningful patterns by showing them lots of data, rather than trying to program in every rule about the world. Taking inspiration from the way neurons work in the human brain, deep learning uses layers of algorithms that successively recognize increasingly complex features — going from, say, edges to circles to an eye in an image.

Notably, these techniques have allowed researchers to train algorithms using unstructured data, where features haven’t been laboriously labeled by human beings ahead of time. It’s not a new concept, but recent refinements have resulted in significant advances over traditional AI approaches.

Yoshua Bengio, a computer science professor at the University of Montreal, organized a deep learning workshop at the Neural Information Processing Systems conference where DeepMind presented the Atari paper.

Bengio said DeepMind was essentially using deep learning to train software to solve problems even when feedback is indirect and delayed. For the paper, DeepMind trained software to play video games without teaching it the rules, forcing it instead to learn through its own errors and poor scores.

Bengio used an analogy to explain: It’s easier for a student to learn when a teacher corrects every answer on a test, but DeepMind is trying to get machines to learn when the only feedback is the grade.

“It’s a much harder problem,” Bengio said. “But there are lots of problems in the real world that are like this.”

Fear That Winter Is Coming

The ramping excitement — and the head-turning size of the Google-DeepMind talent acquisition deal — has made some in the artificial intelligence space concerned.

AI folks tend to be a skittish and skeptical breed. That’s because the promise of AI — that machines could be as smart as humans — is still more science fiction than reality. And hype has spiraled out of control multiple times over the past 75 years, with repeated letdowns famously leading to so-called “AI winters,” when funding and interest went cold for years at a time.

In particular, the notion that DeepMind asked that Google create an internal ethics board as a condition of the acquisition, as reported by The Information, had some AI researchers griping.

Google declined to comment on speculation about the creation of an ethics board.

It’s strange to imagine that trying to hold a giant company like Google to an ethical standard would be a cause for concern. (C’mon — any modern moviegoer is familiar with the specter of robots taking over the world. Of course there are ethical issues present.)

But some in the AI research community think that’s something that can be dealt with at a more realistic date.

“Things like the ethics board smack of the kind of self-aggrandizement that we are so worried about,” one machine learning researcher told Re/code. “We’re a hell of a long way from needing to worry about the ethics of AI.”

Of course, other people see the value of an ethics board.

Patrick Lin, director of the Ethics + Emerging Sciences Group at California Polytechnic State University, said an ethics board should merely raise the potential issues that software designers might not consider in designing AI tools.

“I don’t see the board’s job being to say ‘you can’t do this and can do this,’ but rather ‘let’s look at … how we can mitigate the risks and address the possible harms,’” he said.

To DeepMind’s credit, it was not exactly an epicenter of hype. Prior to being bought, the company was only really known to people who were recruited by it or had colleagues who were. In a 2011 interview that predated DeepMind, co-founder Shane Legg said he gave only a 50 percent chance that human-level machine intelligence would exist by 2028.

Search, Knowledge and Kittens

So what will Google do with DeepMind? Artificial intelligence is core to many teams at Google, from the self-driving car to the search results page.

Jeff Dean (the Google executive running the team that DeepMind is joining) was the lead author on a paper in 2012 that boasted of training a deep network “30 times larger than previously reported in the literature” for the purposes of large visual object recognition tasks and speedy speech recognition. He also worked on a somewhat famous project where a neural network of 16,000 computers presented with stills from 10 million YouTube videos taught itself to recognize cats.

That project was originally part of the secretive Google X research lab, but was later incorporated into more core search work, John Markoff at the New York Times reported.

“Google uses machine learning in every nook and cranny of what they do,” said Pedro Domingos, a computer science professor at the University of Washington. “Larry Page and Sergey Brin don’t say it, but they want to solve the AI problem. They really do want AI to come true.”

Google has been buying up companies and hiring leading researchers in the artificial intelligence space for years, including Ray Kurzweil, Sebastian Thrun, Peter Norvig and Geoffrey Hinton. The acquisition of the DeepMind team adds co-founder Demis Hassabis to that lineup, who worked as a neuroscientist before moving into AI.

Hassabis has closely studied how the brain functions — particularly the hippocampus, which is associated with memory — and worked on algorithms that closely model these natural processes.

“He is serious about combining neuroscience and machine learning, which is a very hot and very promising area,” Domingos said.