Friendly Artificial Intelligence - Criticism

Criticism

One notable critic of Friendliness theory is Bill Hibbard, author of Super-Intelligent Machines, who considers the theory incomplete. Hibbard writes there should be broader political involvement in the design of AI and AI morality. He also believes that initially seed AI could only be created by powerful private sector interests (a view not shared by Yudkowsky), and that multinational corporations and the like would have no incentive to implement Friendliness theory.

In his criticism of the Singularity Institute's 2001 Friendly AI guidelines, he suggests an AI goal architecture in which human happiness is determined by human behaviors indicating happiness: "Any artifact implementing 'learning' must have 'human happiness' as its only initial reinforcement value and 'human happiness' values are produced by an algorithm produced by supervised learning, to recognize happiness in human facial expressions, voices and body language, as trained by human behavior experts." Yudkowsky later criticized this proposal by remarking that such values would be better satisfied by filling the Solar System with microscopic smiling mannequins than by making existing humans happier.

Ben Goertzel, an artificial general intelligence researcher, believes that Friendly AI cannot be solved with current human knowledge. In the past he has stated that he does not believe mathematically proven Friendliness to be possible. In 2010 Goertzel favored formulating a theory of AI ethics "based on a combination of conceptual and experimental-data considerations" by " early-stage AGI systems empirically, with a focus on their ethics as well as their cognition". As of 2011 he proposes to build an "AI Nanny" system "whose job it is to protect us from ourselves and our technology – not forever, but just for a while, while we work on the hard problem of creating a Friendly Singularity."

Adam Keiper and Ari N. Schulman, editors of the technology journal The New Atlantis, argue that it will be impossible to ever guarantee "friendly" behavior in AIs because problems of ethical complexity will not yield to software advances or increases in computing power. In particular, they criticize Yudkowsky's definition of Friendliness, noting a variety of situations (such as hostage scenarios) in which it would dictate that a Friendly AI should simultaneously take conflicting or opposite actions. They write that the utilitarian calculi upon which Friendly AI theories are based work "only when one has not only great powers of prediction about the likelihood of myriad possible outcomes, but certainty and consensus on how one values the different outcomes. Yet it is precisely the debate over just what those valuations should be that is the stuff of moral inquiry.... Simply picking certain outcomes — like pain, death, bodily alteration, and violation of personal environment — and asserting them as absolute moral wrongs does nothing to resolve the difficulty of ethical dilemmas in which they are pitted against each other."

Stefan Pernar argues along the lines of Meno's paradox against the usefulness of Yudkowsky's approach by pointing out that attempting to solve the FAI problem is either pointless or hopeless depending on whether one assumes a universe that exhibits moral realism or not. In the former case a transhuman AI would independently reason itself into the proper goal system and assuming the latter, designing a friendly AI would be futile to begin with since morals can not be reasoned about.