Is there evidence your god beliefs are true while other god beliefs are false?

[Mobyseven]

We are not that far apart here.

Subjective experience was what I had in mind in reference to the medical research. But the real difference lies in someone telling you something, which is simply unreliable, and systematic collection of anecdotal evidence usually with controls of some kind that make anecdotal evidence of value.

First, the definition of 'evidence' includes anecdotes. Again, as with Piscivore and Beth only defining evidence in terms of what it is evidence for or of, you may be doing the same in reverse when you say an anecdote is not evidence. It is evidence. It may not be evidence OF the thing described in the anecdote. So in that sense it might be unreliable evidence, and it certainly isn't evidence of something, unless it turns out to be a valid anecdote. But it is correct to call something anecdotal 'evidence'.

Systematically collecting anecdotal evidence is most often used to evaluate subjective experiences such as pain. But really, anytime a study relies on subjects and controls to relate historical evidence of any kind, that is anecdotal evidence. So, for example, if I want to look at people who successfully lost weight and kept it off and compare them to people who tried to lose weight but were not successful, I might administer the same questionnaire to the two groups about what they did and compare the groups. That would be systematically collecting anecdotal evidence not related to subjective experience. What makes the data useful is having two groups to compare. If it turns out, as it does in the case I am using as an example, that the group who lost weight and kept it off did 3 things differently than the group failing to lose weight, and all the other answers in the questionnaire were answered similarly, then it is valid to conclude those three things were indeed relevant.

(BTW, the three things were, 1) weighing oneself daily, 2) counting calories consumed, and, 3) increasing one's activity level, (in particular, fidgeting more, walking more, and things which increased activity all around like taking the stairs rather than the elevator.)

So you can see that anecdotes themselves are not the problem. It is single anecdotes which are unreliable while systematically collected anecdotes with controls can be very useful evidence in some studies.

Interesting. Again, I don't think we really disagree here in terms of what can be considered evidence - we disagree as to why it is considered evidence.

A single anecdote, as you pointed out, is unreliable. As are a large number of anecdotes, if not collected systematically and controlled. However, I would still not call a single anecdote 'evidence', because there is no way to tell if it really is evidence at all. Any particular anecdote can be evidence for anything, and is therefore really evidence for nothing - and if something is evidence for nothing, can we really be justified in calling it 'evidence' at all?

To return to a slightly different point (which hopefully will better explain my position), think of the weight loss study you mentioned. The anecdotes were systematically collected through a questionnaire, controlled for, and analysed. No one would suggest that due rigor hadn't taken place, nor that the results were worthless or meaningless.

But the anecdotes, in a sense, existed before the participants filled out the questionnaire - they had already had the experience. Were a participant to have relayed their experience to another person prior to filling out the questionnaire, it would have been worthless as evidence - evidence for nothing, and so as I said above, not evidence at all.

And yet, a number of these worthless anecdotes, systematically collected in the right way, can produce meaningful results. Does this mean that the process of systematically collecting and controlling the anecdotes somehow changes their nature - that the anecdote itself becomes somehow meaningful in light of the study?

I would say no, and my reason is as follows: Say that a participant involved in the study tells a person his anecdotal experience after the completion of the study. Is the anecdote worth any more now than it was prior to the completion of the study? No - it would still be wrong to consider the anecdote evidence (again, as per above), as the only way to justify the anecdote would be in light of the study, and once one has the information from the study the anecdote is entirely irrelevant once again.

So if the anecdote cannot be considered evidence (as per above) either prior to the completion or after the study, what makes it valuable? The answer is not the anecdote itself, but instead the controlled systematic collection of the anecdotes! It is the data that exists after the anecdotes have been collected and controlled for that is the evidence, and not the anecdotes themselves. Only in conjunction with the methodology can we derive evidence from the anecdotes. Remove the methodology and the anecdotes are worthless.

It's a subtle difference in our positions - I hope I have managed to make myself clear.

 

[Skeptic Ginger]

I see what you are saying Moby and agree with a caveat. The evidence as you say is actually the act of what the people are reporting occurred. So in that sense the actual evidence is what occurred not the report of what occurred.

However, in medicine, we actually need to use the patient history as the actual evidence. Because we cannot observe directly in retrospect. When a patient comes to see me, I collect a history. I also take that history for what it is, it is a memory and an interpretation. By that, it is flawed. But I cannot observe what the patient is telling me so I actually have to consider all the factors that are involved in the history the patient relates.

I make the diagnosis based on what I observe and on what the patient tells me. I do not make it based on what the patient actually experienced.

This is easiest to demonstrate with a child's medical report. If a child tells me they had a stomach pain, I know that could mean they had chest or abdominal pain. So the evidence is not what occurred because I cannot obtain that evidence. The evidence in this case is what the child actually says. I would look for chest, GI and GU tract problems.

In this case, it isn't just subjective pain, it isn't the actual pain event, it is the specific things a child says in answer to a question that I use as the evidence. And maybe because this is so common in medicine and in medical research, I can see the distinction fairly clearly.

Another example is what we call a 'positive review of systems'. That is when the patient says yes to every symptom you ask if they have. That 'anecdote' is a classic patient history and tells you something about the diagnosis. In these cases I am actually using the anecdote as the evidence.

For another example, when we do retrospective research, a lot of trouble is gone to to validate a questionnaire. So the patient is reporting on the thing that happened. And the thing that happened could be seen as the actual evidence. But because memory of what happened is so subject to interpretation and selection, we actually study the responses, we measure the responses, and we compare the responses. We consider it indirect evidence. It is more than just an anecdote of what happened.

 

[Mobyseven]

I'm too busy with uni, work and other stuff right now to properly address the use of anecdotal evidence in medical diagnostics, but I wouldn't mind continuing the discussion at a later date - if this thread is still active or if we start a new one. Sorry - it's certainly been interesting, but threads such as these (y'know...where I actually have to think ) take a fair bit of time, which I don't have right now.

I would say, just to give you an idea of my position, that I see medical diagnostics as containing inbuilt error checking mechanisms, and also that for many medical conditions evidence other than the patient's complaint must be used to validate the anecdote.

 

[Skeptic Ginger]

No hurry, Moby, answer when you have time. But don't confuse the validity or reliability of the evidence with the evidence. This is what I have been adamantly posting about here for the last dozen or so pages.

The Evidence: the thing, it's properties, whatever about the thing that one is using to determine something from.

The Observation: the observation, measurement, or whatever means one has to obtain the information from the evidence.

Communicating: communicating the observation, measurement, etc., which is actually separate from the observation itself.

The Conclusion: the thing one determines from the evidence, including concluding you cannot draw a conclusion from the evidence, or you can only say the evidence suggests something, etc.

These really are distinct entities despite Piscivore's belief the term, 'evidence', requires one include all the things together. You can see the confusion that causes in these discussions. If one just recognized the parts here, we wouldn't be having such discrepant opinions. We'd likely find with a few exceptions that we mostly agreed with the underlying principles.

I tried to describe to you how the anecdote in the medical history actually was the direct evidence even though it is a patient relating indirect evidence. We take into consideration lots of things that come directly from the anecdote such as the 'positive review of systems' evidence. Other times, you are correct, the evidence was collected by the observer (whether they are a formal or informal observer) and the information about the evidence is then communicated in the questionnaire. In that case we are using the patient as a proxy observer. We may take into account such things as the reliability of patient's memory, observation skills, and so on. But the person answering the questions is, in essence, a proxy observer.

In the case of the patient's medical history, it is a combination of proxy observer and the medical provider's direct observation of the anecdotal account.

The conclusions drawn from the anecdotal evidence is a separate matter.