AlBell
Philosopher
- Joined
- Mar 28, 2009
- Messages
- 6,360
Nor did I mention explicitly nor implicitly a 'gigantic "if/then" block'. Parallel processing is possible and not a new concept.First of all, just because code is written using branch instructions does not imply it is a gigantic "if/then" block. Conceptually, the operation of your neurons is the same as an "if/then" statement, yet multiple neurons acting together certainly produce results that you would not consider a gigantic "if/then" block.
Thanks. IIRC neural nets need to be trained by repeated data inputs to set specific variable values. Whether by node or by multiple nodes I don't know.So, a trained artificial neural network is NOT a look-up table or if/then block or a set of database lookups. Information flows through an artificial neural network exactly like it does through a biological one -- if you tried to look at an individual neuron and make sense of it, you would have no chance. There isn't some variable in the neural network code that corresponds to anything you could make sense of at the level of the entire system.
Furthermore, you can't train neural networks by programming them. You program the code for the nodes ( neurons ), throw a bunch of them together, and then supply the system with repeated instances of something it is supposed to react to ( more or less ). What the system does when it "learns" is beyond your understanding -- the edge weights and behavior of the individual nodes is now out of anyone's hands. At this point it is genuinely no longer just "programmed" because the programmer has literally no idea whatsoever what any of the edge weights or individual node behavior means.
No problem. I'm happy with what you are providing here. Others might appreciate more from you though.I don't know that Watson makes use of neural networks, though. I was just elaborating on one issue I saw with your language.
I can't find much information on the exact software architecture used by Watson, but based on some diagrams ( like the one wikipedia has ) for the high level flow I can guess how I myself would have programmed it to function if my boss told me to use that flow. However it will take some time to get it all together, gimme a bit.
Ok, that makes sense to me. Thanks again."multiple drafts" refers to the idea that instead of an "if/then" type information flow, you have sub-modules that add their little "piece" to a "draft" representing either an incoming percept or an outgoing action ( which can be totally "imagined," meaning the percept doesn't come from external reality and the action doesn't go to external reality but rather come from/go to somewhere internal to the system ) that is then "evaluated" by the "core" module of the system, which in terms of consciousness is our conscious train of thought. This "core" does something with the "draft," adds "feedback" to the draft, and another iteration is performed. Thus this paradigm treats consciousness as a "core" that is "aware" of only the "drafts" being given to it, constantly churning over them, and spitting them back into the lower levels of the system with some feedback.
Thanks again for helping me out. I plead wrongness on my table-lookup idea."global workspace" refers to the idea that instead of an "if/then" information flow, you have sub-modules with access to a shared workspace of information, and when they see something that "interests" them they act on the information and then spit a modified version of it back into the workspace. There is also a "core" module, which again is the "conscious awareness" part, that monitors the shared workspace for anything it finds interesting. When such a thing is there, it becomes aware of it, does something to it, and spits it back into the shared workspace.
Note that in both of these models the "core" isn't necessarily something "core" to the functioning of the system, it could also just be whatever is "consciously aware" of stuff. It is obvious that we humans can do many complex tasks without really being conscious of them, or certainly that the actions in those tasks are not front and center in our conscious train of thought.
The fascinating thing about both of these models is that 1) they work really well with neural networks and 2) they align very well with the way human thought seems to occur.
For example, in the global workspace model, the following information flow makes sense:
0) someone turns around and there is a person there, so photons from that face hit the retina of the observer
1) percepts come in from the world and get placed in the global workspace after being filtered through our visual processing system
2) sub-modules look at the individual percepts and see if they can "aggregate" them into more complex percepts, if they can, they spit those more complex "aggregated" percepts into the workspace for other modules to then use.
3) at some point a sequence of sub-modules has built up a piece of information that the core module might recognize as "a person." Since this is sitting there in the global workspace the core module eventually recognizes that information.
4) the core module reacts to the "face" information somehow, perhaps by injecting information into the shared workspace that the body should respond to the face
5) sub-modules look at this "should respond" action and decompose it into possible actions, like speaking, waving a hand, whatever, and inject information about those smaller actions back into the shared workspace
6) maybe the core sees the possible responses and weights one higher than others, or maybe this is done by some other sub-module, in any case, a winning action is selected and put back into the shared workspace
7) this winning action is further decomposed into atomic actions by other sub-modules and the information is sent to the body, where it reacts properly ( by speaking or waving a hand or whatever ).
Now think about just how close this is to our mechanism of operation -- when you see a person, do you see the individual parts of their body? No, of course not, your conscious awareness just recognizes them as a person in one fell swoop. Where did that aggregation come from? And when you respond to seeing a person, do you consciously decide what shapes your larynx should make as you vocalize? Do you decide how to move your arm to wave? Of course not, you just sort of "do" it and some sub-levels of your brain that don't normally pay attention to decompose those high level actions into the atomic steps that your body should take to satisfy them. Do you even decide to "respond" when someone says "hello" or is it sort of automatic? This model accounts for why all of those things happen the way they do.
Finally, "associative mapping" refers to associative memory, which is formally known as content-addressable memory. This is memory that instead of addressing it by address, like in computer memory, you give it some piece of "content" and it "gives back" the thing it remembers that is "closest" to the input. Not surprisingly, neural networks use content-addressable memory.
Some fascinating things about such neural networks ( you can google "hopfield network" since that is the most famous kind ) -- the closer the "input" is to the actual "memory" the faster the system will converge on that memory ( sounds like your memory, doesn't it ? ) and the more "memories" the system has been trained on the longer it takes to converge ( sounds like your memory, doesn't it ? ) and finally if the system is loaded with memories past some threshold number ( which is related to how many nodes the network has ) there is a higher chance of either converging on the wrong memory or not converging at all ( sounds like your memory, doesn't it ? ).
One final note, which you hopefully find extremely fascinating, is that you can use content-addressable memory to perform a sort of implicit logical inference by varying the input. For example, when someone asks you "name some things that are red," what is the process your mind uses? I won't poison the question, but what you can do with content-addressable memory is first just throw "red" at it and see what comes back, and then start throwing multiple things at it, like "red vehicle" or "red fruit" or "red painting" and see what comes back, and then go even further with things like "red rothko painting" or "red sunset," and when you put stuff like that in, the system might spit back out memories such as "I saw that red rothko painting at the Dallas museum of art" or "the prettiest sunset I ever saw was when I got that flat tire near El Paso" because those things are actual memories in the system.
I say that is "implicit inference" because logically the state "red car" does have a connection to "that red audi I saw yesterday," and you can easily answer logical questions like "have you ever seen a green audi" by just coming up with zero memories of such a thing. No, I have never seen a green audi -- and I didn't even need to parse the sentence like a computer such as Watson would need to parse it, I can rely on my associative memory to give me the logically correct answer.
As an aside, a tv show on brain function implied strongly that only a few neurons were needed to store face images, and that's why face recognition is often very rapidly accomplished by humans.