Doesn't this come down to what emotions are programmed in?
That's part of the problem. Emotions seem to be required, in humans, to learn and categorize experiences.
However, it isn't what I'm getting at.
What I'm getting at is the idea that any system complex enough to be capable of consciousness is complex enough to get stuff wrong and have internal conflicts. Just consider the fact that conscious people do philosophy, and hardly any agreement exists there.
HAL's psychosis is explored (at least in Clarke's mind) as resulting from conflicting instructions: to be helpful, accurate, and informative, but also to conceal the purpose of the mission from the astronauts. Interesting idea. Maybe the bomb in
Dark Star is a better exploration, with its focus on phenomenology, but I still have a bad taste in my mouth about how they ripped off the ending from Ray Bradbury.
What I'm getting at is this: Humans, who seem to be conscious, can do things that computers cannot, at least now. I don't know if they are related to emotions, but they may be related so similarly weird stuff, like empathy and self-awareness. It seems to me from a mathematical/logical perspective that self-awareness, which many people think is a
sine qua non of consciousness, evolved as a means of multiplying intelligence. The ability to model something outside the brain in a similar way to modeling something inside the brain or the brain itself, including the whole mirror neuron thingie, seems to me a neat basis on which to build entire categories of learning and reasoning. Generalizing self-to-others and others-to-self is a powerful thing to do, and self-awareness may be a way to go about it.
But with this power comes the expense of making inappropriate generalizations, and do you want your computer to do that? Probably not. I see these commercials with people talking to Siri on their iPhone, and I am absolutely certain that's what people want, and I'm also certain that the commercials are largely fake or rely only on cases where it happens to get it right.
I'm also certain that the requirements are way beyond what you can expect another human being to do. People spend a hell of a lot of time correcting misunderstandings. Just consider what I'm trying to explain now, because I don't think I was understood. People do this all the time but they put up with it because of other things about interpersonal relationships (which do have to do with emotion).
But people seldom study that. I've studied a lot of linguistics, and it constantly irritated me how structural linguists ignore almost all of verbal communication, living in some idealized la-la land that doesn't even come close to modeling what they write in linguistics books and papers.
So, the question is, would you want a computer that is only as good as another human being at understanding you? I'm guessing that the answer would be no, and that this would prevent people from actually thinking along the lines of making a conscious machine. Either it would be too irritating or too scary, and people would flip out before they sat down and wrote the code.
