Chris Carter Exposes Richard Wiseman Misdeeds

[fls]

Linda, since you're appear to be well-read on this I'm surprised you don't know that Sheldrake controlled for this "problem", but running short test and very long ones (some where PS didn't come home until the next day)... the results were the same.

Yes, Sheldrake also looked at non-randomized trials, and a set of trials he called "controlled trials" which took place in the evenings during which Pam did not return. As I already mentioned, the control trials showed the same results - the dog visited the window at the same intervals that he visited the window in the randomized trials. That is, the dog showed same the behaviour regardless of whether Pam was returning or not.

Linda

 

[GnaGnaMan]

We're going in circles... Wiseman was collaborating with Sheldrake... Sheldrake never made this claim (BTW Wiseman's the one who came up the with 10-minute interval thing before the research started than abandon it for the 2-minute debunk test).

Do you have a source for what you claim about Wiseman about the 10-minute interval?

The claim that Wiseman tested is pretty much in:
A Dog That Seems To Know When His Owner is Returning: Preliminary
Investigations

The main difference is that in that paper the owner's parents judge subjectively when the dog starts his waiting behavior while Wiseman and M. Smith came up with an objective criterion for recognizing this behavior.

Moreover, as was pointed out in this thread PS never made this 2-minute claim... and why the heck would you simply try and debunk a scientific experiment based on a casual conversation with one of the participants... not consulting your collaborator.

How do you know it was based on a casual conversation?


The one problem that I see with what they did is that 4 trials is very few but that does not mean that this did not have a reasonable chance of success.
Post hoc you can come up with other valid reasons why they were not succesful but you can't demand that these other reasons should have been anticipated.

 

[alex.tsakiris]

These trials you're thinking of were part of the preliminary tests. There are too many problems with those to draw conclusions.
Didn't Linda mention that somewhere upthread?

incorrect... they were done after Susan Blackmore brought up this critique.

 

[alex.tsakiris]

Yes, Sheldrake also looked at non-randomized trials, and a set of trials he called "controlled trials" which took place in the evenings during which Pam did not return. As I already mentioned, the control trials showed the same results - the dog visited the window at the same intervals that he visited the window in the randomized trials. That is, the dog showed same the behaviour regardless of whether Pam was returning or not.

Linda

Ok, now you're really fudging the data... AND this whole 10-minute interval thing is an artificial creation (by Wiseman). I mean, look at the exact times (not the ten minute intervals) and see if your data mining stuff holds up... spoiler alert -- it doesn't).

Anyone who bothers to read the papers published on sheldrake.org will see thru this, or you can listen to the many shows on skeptiko.com that cover the topic.

Thx for the dialog... happy to continue, but not sure I'll get back to this thread for a while... anyone can post on Skeptiko (first hurdles not withstanding )

 

[alex.tsakiris]

Do you have a source for what you claim about Wiseman about the 10-minute interval?

not off the top of my head, but it is so... I'll send you a skeptiko t-shirt if I'm wrong

 

[GnaGnaMan]

incorrect... they were done after Susan Blackmore brought up this critique.

I do not know when Blackmore originally made her critique or why that matters.

Let me be precise now. Both the paper with preliminary in the title and the one following it contain randomized trials. Both also contain uncontrolled observations.
The preliminary paper relies on the dog owner's parents to write down when return is signalled and claims that the dog knows exactly when the owner is starting to return.
The following paper relied on videotaping the spot where the dog went to signal and no longer made any claim that, or even attempt to test if, the dog knew exactly when the owner started to return.
From the uncontrolled observations in either paper no solid conclusions can be drawn. (I admit, I had forgotten that the videotape paper also contained uncontrolled observations...)

not off the top of my head, but it is so... I'll send you a skeptiko t-shirt if I'm wrong

I think I've found the source for what you mean.

In September 1996, Wiseman and I met to discuss these findings. He raised objections to the way I had plotted the data on graphs, and suggested an improved method, dividing up the experimental period into 10-minutes intervals.

Is that it?

Contrary to what you claimed the suggestion was made almost a year after RW and MS had conducted their experiments. Also a certain way to graph data does not provide a criterion to judge if the dog successfully predicted the return.

 

[Ersby]

Sheldrake never made this claim (BTW Wiseman's the one who came up the with 10-minute interval thing before the research started than abandon it for the 2-minute debunk test).

Which claim? The claim that Jay-tee's anomalous behaviour was that he reacted when Pam began her journey home? Sheldrake did make that claim, in his paper "A Dog That Seems To Know When His Owner is Returning: Preliminary Investigations" Sheldrake, Smart; Journal of the Society for Psychical Research 62, 220-232 (1998)

I thought that by quoting the two relevent sections side by side, that the connection would be clear. Apparently not.

But you're right inasmuch as the "two minute" part of the claim is not part of Sheldrake's original research.

I jumped into this thread to demonstrate how many skeptics are unable to distance themselves from crappy science/methods when it comes from skeptical sources...

My "skeptical" source in this case being the Journal of the Society for Psychical Research!

y'all have proven my point.

Really?

 

[fls]

Ok, now you're really fudging the data... AND this whole 10-minute interval thing is an artificial creation (by Wiseman). I mean, look at the exact times (not the ten minute intervals) and see if your data mining stuff holds up... spoiler alert -- it doesn't).

I used exact times for Wiseman, although the exact start times are not given, so the elapsed time (time since the owner left) might be off by a minute or two. This would make no difference to the analysis, since the dog was not asked to indicate when Pam left down to the exact minute. Sheldrake plots elapsed time by 10 minute intervals so his data is a bit more granular, but this doesn't matter (and note, it is only the randomized returns which are relevant, since normal returns will include all the non-psychic ways in which the dog's behaviour is influenced) since we don't really expect the dog to only go to the window exactly 50 or 120 minutes after Pam has left.

Anyone who bothers to read the papers published on sheldrake.org will see thru this, or you can listen to the many shows on skeptiko.com that cover the topic.

I don't see how you could have understood my concerns with your criticism, since I did use Wiseman's exact numbers and Sheldrake's exact numbers are unpublished. You must be talking about some unrelated information instead of addressing what I have been talking about. All I am pointing out, which is there for everyone to see in Sheldrake's paper, is that the dog spends more time at the window after Pam has been gone for a bit less than an hour, after she has been gone for about two hours, and after she has been gone for about 2-1/2 hours. On top of this, he is also more or less active for other reasons (traffic on the street, time of day). When Pam is told to return after two hours and after 2-1/2 hours, it leads to a spurious association with behaviour that was occurring in the absence of her return.

Linda

 

[alex.tsakiris]

I used exact times for Wiseman, although the exact start times are not given, so the elapsed time (time since the owner left) might be off by a minute or two. This would make no difference to the analysis, since the dog was not asked to indicate when Pam left down to the exact minute.

What do you mean ? This shows the problem with your analysis... the dog wasn't asked anything! But a 1 or 2 minute difference in the times when Jay-tee went to the window kinda crushes your theory.

All I am pointing out, which is there for everyone to see in Sheldrake's paper, is that the dog spends more time at the window after Pam has been gone for a bit less than an hour...

It's there for everyone to see, but you're analysis is just plain wrong:

http://www.sheldrake.org/D&C/controversies/wiseman.html

In the light of Blackmore's comments, I have reanalyzed the data from all 12 experiments excluding the first hour. The percentage of time that Jaytee spent by the window in the main period of Pam's absence was actually lower when the first hour was excluded (3.1%) than when it was included (3.7%). By contrast, Jaytee was at the window 55.2% of the time when she was on the way home. Taking Blackmore's objection into account strengthens rather than weakens the evidence for Jaytee knowing when his owner was coming home, and increases the statistical significance of the comparison. (Including the first 60 minutes of Pam's absence in the analysis, by the paired-sample t test, t=-5.72, p=0.0001; excluding the first 60 minutes, t=-5.99, p<0.0001.)

 

[fls]

What do you mean ? This shows the problem with your analysis... the dog wasn't asked anything! But a 1 or 2 minute difference in the times when Jay-tee went to the window kinda crushes your theory.

One or two minute differences aren't really relevant to these analyses. Sheldrake asks whether the dog spends more time at the window after Pam leaves for home than he does before Pam leaves for home. The exact time doesn't matter, because it turns out that the dog spends increasing amounts of time at the window based on the elapsed time for most time intervals preceding the 120 and 160 minute marks. If you look at the plot of the control data (figure 5), at almost any arbitrarily chosen point between 90 and 160 minutes of elapsed time, the dog will spend more time at the window in the 10 minutes after that point than he will in the 10 minutes before. So there is a 70 minute period where Sheldrake's hypothesis (that the dog visits the window more when the owner is returning) will be 'confirmed' even though the dog's behaviour has nothing to do with the return of the owner. I agree that attempts to account for this on post hoc analyses could make use of exact times. However, a better solution is to use a well-designed experiment, so that this effect does not differ between experimental and control trials.

It's there for everyone to see, but you're analysis is just plain wrong:

http://www.sheldrake.org/D&C/controversies/wiseman.html

In the light of Blackmore's comments, I have reanalyzed the data from all 12 experiments excluding the first hour. The percentage of time that Jaytee spent by the window in the main period of Pam's absence was actually lower when the first hour was excluded (3.1%) than when it was included (3.7%). By contrast, Jaytee was at the window 55.2% of the time when she was on the way home. Taking Blackmore's objection into account strengthens rather than weakens the evidence for Jaytee knowing when his owner was coming home, and increases the statistical significance of the comparison. (Including the first 60 minutes of Pam's absence in the analysis, by the paired-sample t test, t=-5.72, p=0.0001; excluding the first 60 minutes, t=-5.99, p<0.0001.)

Except that this does not address the criticism. Regardless of whether the first hour of messier data is included in the analysis or not, the main pattern of spending increasing amounts of time at the window will ensure that more time is spent at the window after any arbitrarily chosen point than before that point.

Linda

 

[GnaGnaMan]

Mr. Tsakiris gives his view of this discussion in his podcast. Link to a transcript.

Providing a transcript is seriously cool.

It means I can just c/p something that jumped out at me.
Even a lot of the posters on the JREF forum were kind of carefully tip-toeing around like ‘okay, let’s not get too close to that one because we know it doesn’t have any solid ground.’ But you really couldn’t have gotten that out of the forum.

I was not tip-toeing and I don't think anyone else was.
Where does this impression come from?
If "you" could not have gotten that out of the forum then how did Mr.Tsakiris do just that?

 

[fls]

Yeah, that's an odd thing to say. I thought I was pretty clear that I find Wiseman's approach more valid than Sheldrake's.

Linda

 

[Ersby]

So why are all of Wiseman's and Milton's negative Stouffer Z figures wrong?

Apologies for the delay, but I finally got round to looking again at Milton and Wiseman's z scores to see what method they used. Cross-checking the meta-analysis with the original papers, I can see that M&W use the original authors' z-score if it is in the paper. Otherwise, they take the reported p number and convert that to a z-score. (There are a couple that use other statistical measures. Not sure what M&W did there but it looks like they used the hit rate to calculate a p number, and from that a z-score).

As for M&W's mistakes (ie, having a z-score lower than if you scored no hits at all), on those experiments the paper did not use hit rates as the primary measure of success. For example, series 5b used sum of ranks, which is how the z-score managed to be so low.

 

[Rodney]

As for M&W's mistakes (ie, having a z-score lower than if you scored no hits at all), on those experiments the paper did not use hit rates as the primary measure of success. For example, series 5b used sum of ranks, which is how the z-score managed to be so low.

So you're saying that series 5b involved ranking the target picture and three other pictures, and that if the target picture was ranked 4th, that contributed more to a negative Z score than if the target picture was ranked 2nd or 3rd?

 

[Ersby]

So it would seem. Although I've made a mistake. My fault for posting just as I'm about to go out, I suppose.

Looking at series 5b again, I see that the primary measure is "mean ESP score", and they report sum of ranks, too. But M&W use the score from the sum of ranks, which is the lower of the two.

 

[SusanB-M1]

Coincidentally, Chris Carter is Author of the Month on the Graham Hancock forum. I looked at one or two of the topics, but decided not to join in!

 

[Rodney]

So it would seem. Although I've made a mistake. My fault for posting just as I'm about to go out, I suppose.

Looking at series 5b again, I see that the primary measure is "mean ESP score", and they report sum of ranks, too. But M&W use the score from the sum of ranks, which is the lower of the two.

The ganzfeld experiments that M&W analyzed generally used hit-or-miss protocols. If I understand series 5b correctly, there was a non-hit-or-miss protocol, but nonetheless the recipient attempted to guess which of four pictures was the target in each of ten trials. So, in their meta-analysis, why did not M&W simply: (1) add the number of hits in those ten trials and the number of hits in any other experiments that did not use hit-or-miss protocols to the number of hits in experiments that used hit-or-miss protocols; (2) add those ten trials, as well as the number of trials in other experiments that did not use hit-or-miss protocols, to the number of trials in experiments that used hit-or-miss protocols; (3) divide the total number of hits by the total number of trials to determine the aggregate hit rate, and (4) use a binomial distribution to determine whether that hit rate was statistically significant?

 

[Ersby]

The ganzfeld experiments that M&W analyzed generally used hit-or-miss protocols. If I understand series 5b correctly, there was a non-hit-or-miss protocol, but nonetheless the recipient attempted to guess which of four pictures was the target in each of ten trials. So, in their meta-analysis, why did not M&W simply: (1) add the number of hits in those ten trials and the number of hits in any other experiments that did not use hit-or-miss protocols to the number of hits in experiments that used hit-or-miss protocols;

The hit rate isn't in the original paper.

(2) add those ten trials, as well as the number of trials in other experiments that did not use hit-or-miss protocols, to the number of trials in experiments that used hit-or-miss protocols; (3) divide the total number of hits by the total number of trials to determine the aggregate hit rate, and (4) use a binomial distribution to determine whether that hit rate was statistically significant?

But neither did Honorton, Rosenthal, Radin, Bem or Storm. I think your issue here is really with statistics, not skepticism/parapsychology.

 

[Rodney]

The hit rate isn't in the original paper.

But that figure was surely recorded and could have been obtained.

But neither did Honorton, Rosenthal, Radin, Bem or Storm.

Perhaps because they obtained significant results using a similar methodology, they didn't think it was worth arguing about. But my point is that M&W's meta-analysis compares apples and oranges. For example, the worst Z score that could be obtained in a ganzfeld experiment with a hit-or-miss protocol, ten trials, and a 25% probability of success in each trial, would be -1.59 (if there were no hits). And yet M&W use the Z score of -2.06 reported for series 5b using a non-hit-or-miss protocol, which they then aggregate with experiments that used hit-or-miss protocols. Does that seem valid?

 

[Ersby]

But that figure was surely recorded and could have been obtained.

Well, I imagine so, but why would they? They have the results already there in front of them.

Perhaps because they obtained significant results using a similar methodology, they didn't think it was worth arguing about.

Seriously, I think your questions about the methods of meta-analysis would be better answered on a statistics forum. You seem to be saying that all these people knew they were using the wrong method, but didn't mind since they got significant results. Is that right?

But my point is that M&W's meta-analysis compares apples and oranges. For example, the worst Z score that could be obtained in a ganzfeld experiment with a hit-or-miss protocol, ten trials, and a 25% probability of success in each trial, would be -1.59 (if there were no hits). And yet M&W use the Z score of -2.06 reported for series 5b using a non-hit-or-miss protocol, which they then aggregate with experiments that used hit-or-miss protocols. Does that seem valid?

They reported a p number, so why shouldn't it be included?