Critique of Strong AI Research
If research into Strong AI produced sufficiently intelligent software, it might be able to reprogram and improve itself. The improved software would be even better at improving itself, leading to recursive self-improvement. The new intelligence could thus increase exponentially and dramatically surpass humans.
Hyper-intelligent software may not necessarily decide to support the continued existence of mankind, and would be extremely difficult to stop. If its utility function were written carelessly, it might become a destructive amoral monster without regard for human values.This topic has also recently begun to be discussed in academic publications as a real source of risks to civilization, humans, and planet Earth.
One proposal to deal with this is to ensure that the first generally intelligent AI is 'Friendly AI', and will then be able to control subsequently developed AIs. Some question whether this kind of check could really remain in place.
Read more about this topic: Strong AI
Famous quotes containing the words critique of, critique, strong and/or research:
“Wagner’s art is the most sensational self-portrayal and self- critique of German nature that it is possible to conceive.”
—Thomas Mann (1875–1955)
“In its artless cruelty, Dallas is superior to any “intelligent” critique that can be made of it. That is why intellectual snobbery meets its match here.”
—Jean Baudrillard (b. 1929)
“You have such strong words at command, that they make the smallest argument seem formidable.”
—George Eliot [Mary Ann (or Marian)
“One of the most important findings to come out of our research is that being where you want to be is good for you. We found a very strong correlation between preferring the role you are in and well-being. The homemaker who is at home because she likes that “job,” because it meets her own desires and needs, tends to feel good about her life. The woman at work who wants to be there also rates high in well-being.”
—Grace Baruch (20th century)