On the nature of evidence

[athon]

This topic seems to crop up every now and then (I think I've started one or two threads in the past on the topic), and I have a real fascination with it. I guess because as a science communicator, and an ex-scientist (not to mention science teacher) I've come to see that so much of belief formation comes down to precisely how one defines 'evidence'.

I'd like to propose my view of it here and see how others feel about the definition.

To me, evidence is any observation which can be used in support of a model which explains a phenomena. As such, a single observation can be evidence for many models for a large number of phenomena.

For example, I see a bright light in the sky. It is evidence of an intergalactic spaceship, a fairy, a plane, a helicopter etc. However, other observations (from the past and the present) are then used to whittle these explanations down to a few models which could lead to a potential
phenomena. Therefore if somebody sees a bright light and claims it is evidence for a flying saucer, they are not exactly wrong. Claiming they have no evidence is therefore incorrect, and unlikely to address the real problem. What they don't have is an appreciation of concepts such as parsimony etc.

The reason I say this is occasionally I see skeptics fall for something of a post hoc rationalisation. For example, I feel that a person seeing a fairy is evidence for fairies. If fairies were real, seeing one would definitely be evidence for it. Yet on determining there are no fairies, some skeptics believe that seeing one cannot possibly be evidence for their existence. Which is nonsense. Seeing a fairy can be evidence for a whole number of things - we determine the most useful model explaining the observation based on combining other pieces of evidence (gathered through a variety of means, such as experimentation) as well as applying concepts such as parsimony.

Any time somebody articulates a statement, the statement could exist on a spectrum of true to false. However, we can only gain confidence in its accuracy based on observations we've had in relation to it, which is in itself evidence.

This relates to recent discussions on 'opinions' which Msr Larsen has been struggling with. I put forward that not all opinions are equal, and that even claims on personal taste could be supported with evidence should the situation ever call for it. Retreating to the statement that a belief is 'just an opinion' does not automatically absolve one of needing to provide evidence (even if requiring evidence on some opinions might be incredibly pedantic).

What is your definition of 'evidence'? Does it only exist in conjunction with what we currently know in the universe? Is seeing a fairy evidence for fairies?

Athon

 

[Francesca R]

To me, evidence is any observation which can be used in support of a model which explains a phenomena. As such, a single observation can be evidence for many models for a large number of phenomena.

Yes I think it can. The strength of a theory though, is how much evidence it can explain, and how inferior other theories are to explain the same observations. The greater the contrast here, the greater the conviction with which the competing theories can be parsed. Also, a good theory probably has to possess (and obviously withstand) falsifying evidence.

For example, I see a bright light in the sky. It is evidence of an intergalactic spaceship, a fairy, a plane, a helicopter etc. However, other observations (from the past and the present) are then used to whittle these explanations down to a few models which could lead to a potential
phenomena. Therefore if somebody sees a bright light and claims it is evidence for a flying saucer, they are not exactly wrong. Claiming they have no evidence is therefore incorrect, and unlikely to address the real problem. What they don't have is an appreciation of concepts such as parsimony etc.

Agreed

I feel that a person seeing a fairy is evidence for fairies. If fairies were real, seeing one would definitely be evidence for it.

Yes it would, but the theory that "there are fairies" may not be supported by much addditional evidence (no other sightings, no transferrable evidence like fairy dust, or an abandoned magic wand), and it may also fail to explain other phenomena.

Yet on determining there are no fairies, some skeptics believe that seeing one cannot possibly be evidence for their existence. Which is nonsense.

Huh? It is evidence for the theory that there are fairies but not convincingly so because it could also be evidence of 1) lying 2) delusion 3) deception etc.

This relates to recent discussions on 'opinions' which Msr Larsen has been struggling with. I put forward that not all opinions are equal, and that even claims on personal taste could be supported with evidence should the situation ever call for it. Retreating to the statement that a belief is 'just an opinion' does not automatically absolve one of needing to provide evidence (even if requiring evidence on some opinions might be incredibly pedantic).

Do you mean opinions ("I like high heels" / "I think John McCain is sexy") or theories ("I think ethanol production is the reason for the rise in food prices")?

 

[I Ratant]

:
I feel that a person seeing a fairy is evidence for fairies. If fairies were real, seeing one would definitely be evidence for it.

Yes it would, but the theory that "there are fairies" may not be supported by much addditional evidence (no other sightings, no transferrable evidence like fairy dust, or an abandoned magic wand), and it may also fail to explain other phenomena.

.
Seeing a fairy proves their existence to that person. But people see fairies and UFOs all the time, and there's still no proof of either.
Where I live, people have seen mountain lions walking thru the neighborhood in the night. No photos though.
There are a couple of indications that these are real sightings.. the people making the reports aren't hysterics, the number of feral cats in the neighborhood has diminished a lot, and, I've taken photos of footprints of very large cats where these lions have been reported.
I believe there are lions in the area, but what they are existing on, other than loose pets is a puzzlement. The neighbors who are aware of the reports don't seem to mind letting their veal...oops. kids, run around in the desert unsupervised where the lions are.
Possibly like a UFO on the White House lawn being the only acceptable proof of one, losing a kid proves there's lions out back.

 

[Francesca R]

Seeing a fairy proves their existence to that person. But people see fairies and UFOs all the time, and there's still no proof of either.

"Proof" sounds a bit absolute here. Consider "evidence" as providing some support that a theory/statement is true.

 

[GreedyAlgorithm]

For me I use "evidence for/against A" to describe E if over random observations R,

[latex]$$ \left| ln \left( P\left(A|X\right)\right) - ln \left(P\left(A|EX\right)\right) \right| >> \left<\left| ln \left( P\left(A|X\right)\right) - ln \left(P\left(A|RX\right)\right) \right|\right>_R$$[/latex]

What does ">>" mean here? It means something like "significant enough that taking note of things like E is a more efficient way to arrive at the truth about A than most alternatives". In reality the right hand side should be "> 0" but in practice we only have a certain amount of computational resources, so taking note of everything is a far worse plan than only taking note of "significant" things.

 

[blutoski]

I find that my use of the word is a bit contextual. Certainly, when I'm engaged in rhetoric or debates with an opponent, I tend to avoid saying "there is no evidence for X". It gives the false impression that I am ignorant of the counterarguments.

What I tend to say is that I'm aware that my opponents have bookshelves of stuff they consider to be good evidence, but that I consider their evidence somewhere between inadequate and downright lame in terms of quality.

At least this supports the impression that I am open minded enough to examine their argument. It also keeps the debate focused on the quality of the evidence and my reasons for dismissing it and may have the side effect of educating observers about skeptical evidence-evaluation methodologies.

 

[Beth]

For me I use "evidence for/against A" to describe E if over random observations R,

[latex]$$ \left| ln \left( P\left(A|X\right)\right) - ln \left(P\left(A|EX\right)\right) \right| >> \left<\left| ln \left( P\left(A|X\right)\right) - ln \left(P\left(A|RX\right)\right) \right|\right>_R$$[/latex]

What does ">>" mean here? It means something like "significant enough that taking note of things like E is a more efficient way to arrive at the truth about A than most alternatives". In reality the right hand side should be "> 0" but in practice we only have a certain amount of computational resources, so taking note of everything is a far worse plan than only taking note of "significant" things.

I've seen this notation used to mean two different things.

1. absolutely continuous
2. Much much greater than


I'm not sure which one is appropriate for your context. There might also be other definitions for it that I'm unaware of.

 

[GreedyAlgorithm]

2. Much much greater than

I'm using number two. ">>" means absolutely continuous? I guess I haven't seen that one before. Wikipedia agrees with you, though.

 

[athon]

Yes I think it can. The strength of a theory though, is how much evidence it can explain, and how inferior other theories are to explain the same observations. The greater the contrast here, the greater the conviction with which the competing theories can be parsed. Also, a good theory probably has to possess (and obviously withstand) falsifying evidence.

I totally agree. However, the acceptance of a theory is not a general group agreement, but rather a personal position. Obviously if a group of people share the same epistemologies, and same understanding of the scientific method, it is more likely they will also accept a theory on the same grounds. However, technically it is a case of a shared personal conviction in the strength of a theory.

Yes it would, but the theory that "there are fairies" may not be supported by much addditional evidence (no other sightings, no transferrable evidence like fairy dust, or an abandoned magic wand), and it may also fail to explain other phenomena.

I agree 100%. Which is why we can agree that it is unlikely that a fairy sighting demonstrates the existence of fairies. However, it is still evidence for fairies (even if it is also evidence for hallucinations, illusions etc).

Huh? It is evidence for the theory that there are fairies but not convincingly so because it could also be evidence of 1) lying 2) delusion 3) deception etc.

I'm not sure what you mean by 'convincingly' so. Not single bit of evidence on its own is necessarily the convincing piece - evidence accumulates and relates with each other. We agree on the second part, though, so I think we're just missing each other's point.

Do you mean opinions ("I like high heels" / "I think John McCain is sexy") or theories ("I think ethanol production is the reason for the rise in food prices")?

Obviously there is a difference there (theories are models which attempt to explain and predict phenomena, while opinions are descriptions of a personal preference), however both, when articulated, become opinions. An opinion is still a personal position, which as I stated earlier so is the personal acceptance of a theory's validity.

Nobody in their right mind would ask for evidence for the latter (unless it appeared significant, such as in a court case or something), however evidence could still be used to support the statement. Yet where the opinion is making a statement on something objective, evidence (observation of an event which supports the opinion) might be needed.

.
Seeing a fairy proves their existence to that person. But people see fairies and UFOs all the time, and there's still no proof of either.

'Proof' is a mathematical term, and has overtones of being absolute in its conclusions. Science doesn't work that way, so I guess I tend to avoid the word proof and go with simply 'evidence' (I notice Francesca says the same thing in her response).

I find that my use of the word is a bit contextual. Certainly, when I'm engaged in rhetoric or debates with an opponent, I tend to avoid saying "there is no evidence for X". It gives the false impression that I am ignorant of the counterarguments.

Good points.

What I tend to say is that I'm aware that my opponents have bookshelves of stuff they consider to be good evidence, but that I consider their evidence somewhere between inadequate and downright lame in terms of quality.

Again, I totally agree. Which is why we need to take care when communicating skepticism that we avoid statements such as 'there is no evidence for X'. Evidence is only useful in the context of other pieces of evidence, and in the context of how strong it is. I feel many people lack the skills in evaluating this weight, lending more credibility to personal anecdote or popular opinion than is due.

The way I see it is this: I ask myself 'if said phenomenom was true, would this observation occur as a direct result of it?'. If the answer is 'yes', it is evidence of that phenomenom. I would obviously then endeaour to seek out other inferences explaining the observation and seek out more collaborating evidence for them.

At least this supports the impression that I am open minded enough to examine their argument. It also keeps the debate focused on the quality of the evidence and my reasons for dismissing it and may have the side effect of educating observers about skeptical evidence-evaluation methodologies.

Cool. I was beginning to think I was alone in this view.

Athon

 

[Dr H]

This topic seems to crop up every now and then (I think I've started one or two threads in the past on the topic), and I have a real fascination with it. I guess because as a science communicator, and an ex-scientist (not to mention science
phenomena. Therefore if someteacher) I've come to see that so much of belief formation comes down to precisely how one defines 'evidence'.

I'd like to propose my view of it here and see how others feel about the definition.

{...}

The reason I say this is occasionally I see skeptics fall for something of a post hoc rationalisation.

I don't see a problem with the definition of "evidence" so much as with a lot of people not being clear on how to judge the strength of one particular kind of evidence as compared with another.

To use your example, if someone tells me they've seen a fairy, that is certainly one kind of evidence. It is strong evidence that this person wants me to believe that they have seen a fairy. On the other hand, it is only very weak evidence for the actual existence of fairies per se, because the person relating the story might be lying, mistakenly interpreting their observation, or delusional.

Put in general terms, anecdotal evidence certainly is a kind of evidence, but it's inherently a weak form of evidence because by itself it is uncorroborated by any sort of replicable correspondence with reality.

What is your definition of 'evidence'? Does it only exist in conjunction with what we currently know in the universe? Is seeing a fairy evidence for fairies?

Evidence is nothing more than an indication of some sort that some particular thing, process, or phenomenon exists in nature. We can have that indication from testimony, from direct observation, or from indirect observation of phenomena suspected to be related to that
thing, process, or phenomenon. The strength of any given form of evidence is determined
by whether or not it has been tested against reality, and if so, how closely it seems to correlate with reality.

Seeing a fairy is one form of very weak evidence in favor of the existence of fairies. Stronger evidence would be multiple observations of fairies under controlled, objective circumstances; positive physical testing on the properties of fairies; or perhaps the capture of a fairy which could then be studied physiologically.

Absent any of this additional corroboration, a single anecdotal report of the observation of a fairy would probably be best explained by either non-fiary hypotheses, or if none of those seem to fit all the facts, by treating it as an anomalous occurance, currently unexplained, and waiting for a replication of the observation and additional data.

 

[CapelDodger]

What is your definition of 'evidence'? Does it only exist in conjunction with what we currently know in the universe? Is seeing a fairy evidence for fairies?

Everything you're presented with is evidence. As Dr H has said, it then comes down to judgement. If someone says they saw a fairy, that's evidence. If a lawyer says his client sees fairies, that's evidence. Neither is good evidence of the existence of fairies, but they're good evidence of other things.

eta : If I saw a fairy I'd cut back on the beer.

 

[arthwollipot]

If athon said that he'd seen a fairy, then I wouldn't count that as evidence in favour of the existence of fairies. If athon, Francesca R, I Ratant, GreedyAlgorithm, blutoski, Beth, Dr H and CapelDodger all said that they saw the fairy, I'd consider it potentially worth investigating further.

 

[Hokulele]

If athon said that he'd seen a fairy, then I wouldn't count that as evidence in favour of the existence of fairies. If athon, Francesca R, I Ratant, GreedyAlgorithm, blutoski, Beth, Dr H and CapelDodger all said that they saw the fairy, I'd consider it potentially worth investigating further.

What if Lost Angeles, a.real.girl, and I all saw a fairy?

 

[arthwollipot]

What if Lost Angeles, a.real.girl, and I all saw a fairy?

Then I'd consider the possibility that cheese fairies exist.

 

[Francesca R]

. . . [snippity] . . .
Athon

Darnit, I agree! Discussions with sensible posters never get anywhere.

 

[athon]

If athon said that he'd seen a fairy, then I wouldn't count that as evidence in favour of the existence of fairies. If athon, Francesca R, I Ratant, GreedyAlgorithm, blutoski, Beth, Dr H and CapelDodger all said that they saw the fairy, I'd consider it potentially worth investigating further.

But that's putting the conclusion before the evidence. It's starting with the assumption that fairies don't exist and therefore the evidence can't be in support of it. If fairies did exist, seeing one would be evidence for it, surely?

Now, it is incredibly unlikely that fairies exist, however if science is about accumulation of relative evidence then claiming something is not evidence simply because we have already concluded it isn't real is putting the cart before the horse.

I guess it comes down to agreeing whether an observation can only be evidence for one thing or for many other things.

Athon

 

[athon]

Darnit, I agree! Discussions with sensible posters never get anywhere.

Haha, sorry. I could be more Larsenesque, but then I'd probably have to kill myself.

Art's putting forward a good point, though, that could produce some interesting discussion. It's not a dead thread yet.

Athon

 

[Hokulele]

But that's putting the conclusion before the evidence. It's starting with the assumption that fairies don't exist and therefore the evidence can't be in support of it. If fairies did exist, seeing one would be evidence for it, surely?

You seem to be putting the conclusion before the evidence the opposite way with your, "If fairies did exist". I would argue that the fairy sighting would be evidence against the conclusion that they do not exist, rather than evidence for the conclusion of their existence, if this makes any sense.

 

[arthwollipot]

I'm confused. Epistemology makes my head hurt.

 

[Ivor the Engineer]

For me I use "evidence for/against A" to describe E if over random observations R,

[latex]$$ \left| ln \left( P\left(A|X\right)\right) - ln \left(P\left(A|EX\right)\right) \right| >> \left<\left| ln \left( P\left(A|X\right)\right) - ln \left(P\left(A|RX\right)\right) \right|\right>_R$$[/latex]

What does ">>" mean here? It means something like "significant enough that taking note of things like E is a more efficient way to arrive at the truth about A than most alternatives". In reality the right hand side should be "> 0" but in practice we only have a certain amount of computational resources, so taking note of everything is a far worse plan than only taking note of "significant" things.

What do X, EX and RX represent?