Here's another little peek into reality, and how it differs from sims.
Let's say I want to teach my son about the interrelationships involved in seal populations in the wild.
We use a computer to simulate this.
We start out very simply. We just have numbers representing seal populations, the populations of various species of fish, the number of fishermen and their harvests, the temperature of the ocean, and so forth.
We set up the relationships, and we can see how the seal populations fluctuate depending on the fish populations, predation, environmental conditions, and such.
It's just numbers, but I can tell my son which numbers stand for what, and we can see how it all works.
My son gets enthusiastic about it, and over time we ramp up the details. We introduce very detailed information about ocean currents, disease in the fish, even how employment rates cause the number of fishermen to change, and even down to how shortages in raw materials cause some boats to be added to the mix or taken out.
We go on from there, adding graphics, making it holographic in fact, representing everything in the system right down to the cellular level. We add projectors and you can actually step into this world and it's as real as if you were there.
In this simulation, there are numbers representing the molecules in each component. It's mind-bogglingly complex. It's so accurate that you can use it to make predictions about the real world.
And yet, despite all this, the computer that's running the sim never catches a fish, never swims, never gets divorced when a collapse of the fish population puts it out of a job, never lobbies Congress for a change in regulations, never freezes in winter.
In other words, the behavior of the computer continues to be computery -- never fishy, never watery, never windy, never sealy, never boaty, never fishermany.
Why? Because we haven't changed the computer into a seal. Or anything else.
We get tired of this eventually, and decide to look into the human body.
We begin with numbers representing blood pressure, height, weight, heart rate, and all sorts of other things.
Over time, we get more and more detailed, until there's nothing we can't know about our simulated human.
But the mechanism of the computer never gets a blood pressure, never runs through a shopping mall to catch up with its dad, never does anything that a human actually does... including being conscious.
There simply is no magical miracle moment at which a sim becomes sufficiently detailed that the machine running the sim stops acting like a machine running a sim and starts acting like the things which we imagine when we look at the outputs of the sim.
And that's why you can't get a machine to be conscious unless you build it to do the same physical stuff that makes a human being conscious.
You can't program your way into it.
There's no reason why there can't be conscious machines. But there's no such thing as a conscious program.
