Using the "it's only anecdotal" strategy here seems a bit bizarre.
How could you do a non-anecdotal experiment in this area?
Of it's nature the data in this area has to be anecdotal.
You raise a good point. However, I utterly disagree with you. I can think of several ways to come up with real evidence (not anecdotal observations) if reincarnation were a fact.
First, I think reincarnation advocates have the cart before the horse. They have an idea and are looking to support it, rather than following observations and looking at all the available evidence.
If reincarnation arguments aren't merely apologetics (where they start with a belief, then try to build a case to support that position), then the only "phenomenon" the theory of reincarnation is trying to explain are these children's stories, which are collected as anecdotes.
As PlumJam correctly points out, that's really the only way these stories would ever come to our attention. Surely we couldn't take a given number of children and isolate them from confounding influences until they're old enough to talk and then see what sort of past life memories they've got--and with what frequency etc. So there's no way to know how much of these stories were "fixed" memories (supplied by prompting from the family, confabulation, outright fabrication, etc.)
So, if the question is how best to explain these stories, I've already pointed out that Occam's Razor strongly prefers the mundane explanations. But, that's not evidence, that's just pointing out that if you did find evidence of reincarnation, you'd have to overhaul an awful lot of what we know about neuroscience, memory, language, etc.
Back to PJ's question, "How could you do a non-anecdotal experiment in this area?"
One way is to test one of these kids. You'd have to come up with several bits of information that are not likely to be known by the kid or the family (or the neighborhood) etc. Then, ask the kid questions based on those bits of information. If he can remember some details about a past life, we should expect him to remember others. The problem with anecdotes is that much is made of a "hit" with no consideration of a great many "misses".
I know there are stories of similar tests being done with reincarnations of holy men (where the kid is supposed to pick out possessions that belonged to the deceased llama or whatever), but those can hardly said to be done in controlled circumstances. (I tend to doubt the veracity of their retelling since those who profess them have pretty strongly vested interests in proving that the kid is the reincarnation.) At least they're on the right track. It is most certainly possible to come up with non-anecdotal tests of the kids' stories.
Another line of non-anecdotal evidence is related to the stuff I brought up earlier. If there's a disembodied something that holds memories of past life, we should expect some brain structure to interact with that. If reincarnation were real, we could presumably find such structures and maybe even detect and measure that disembodied something.
Granted, absence of this evidence doesn't prove that the thing doesn't exist, but this is a type of non-anecdotal evidence that would be possible. This is the sort of thing it would take for me to reconsider my position. You have to admit that nothing exists except anecdotal evidence.
Memory is very plastic, and that's really all we're dealing with here. There are no controls on the information the kid might have been exposed to from other sources.
The mundane explanation that I accept (in the absence of any other evidence) is that in a culture where a lot of people take reincarnation as an assumed fact, you have a toddler doing some random behavior. Someone in the family or community thinks that behavior is similar to a behavior or mannerism of someone recently deceased, and starts talking about the kid being the reincarnation of that person. From there, everyone begins collecting (or inventing) proof of that. (No one considers the great many behaviors the kid does that look NOTHING like the supposed previous life.) The accounts of what the child actually did or said get lost in the developing story that gets spread around. At the same time, the kid is pretty much taught to "remember" (it's very easy to implant false memories) events from that person's life.
With this explanation, no disembodied thing that can somehow store memories is required. No brain structures that somehow upload and download these memories (or otherwise interact with the disembodied thing) are required. There are no conflicts with what is known about memory, language, brain function or neuroscience in general. It's much more parsimonious.
So if you weigh ALL the evidence, it points to the mundane explanation. As a skeptic, that's the one I provisionally accept.