Michael Shermer vs Jeffrey Armstrong

[EHocking]

Again a fast reply:

I don't usually do this, but kudos for following this up - and points to Michael Shermer for getting back to you on it.

The SkeptiWiki entry for Shermer could potentially add a bit of "controversy" to his comments, though as it states that, "Shermer also produces and co-hosts the 13-hour Fox Family television series,".

Which would seem to contradict his comments that he had no control over the final edit - except that the TV IMDB entry notes that he was NOT the producer, so the Skeptiwiki entry appears to be incorrect.

Certainly won't help him "clear his name" over the video clip being touted by bleevers.

 

[contingent]

According to -- http://www.petemccormack.com/blog/?p=339 --

"Further, Michael Shermer, seemingly caught off guard by Jeffrey’s accuracy, supposedly delivered an ending so irritable and unbalanced that Jeffrey called the show’s producer and pointed out that it was 'out of integrity' for them to not point out that the experiment had gone completely in favour of astrology.

"The producers, to their credit, made sure the ending was changed.

"But even after Jeffrey’s statistical success under the worst possible conditions, Michael’s closing statement rose only to:

"While skeptics will explain the results of our study as due to chance and wishful interpretation, believers will see them as further proof that the stars and planets directly influence our lives.

"One can only imagine the lambasting that would have resulted had the last two 'double blind' readings unfolded with opposite results. The term 'tarred-and-feathered' clucks to mind."

The short story is this: we ran out of time at the end of the filming day to conduct any more experiments with Armstrong. I protested that it was going to make it look like he was successful, but to no avail as I did not have final authority over what was produced for the show, Exploring the Unknown, and so I just hoped that in the editing process it would be cut in a way that dealt with that problem, but it wasn't and I couldn't do anything about it, so it aired and no one noticed back then (in 2000), but someone posted the clip you reference and now we're dealing with the fallout from it. It is an unfortunate reality of the series that I didn't have enough control over the production and filming process.

You can post this explanation if you like.

Michael

Wow I wonder who was lying lol? Either their was a second part or not it seems like there wasn't therefor pat is laying.

 

[JoeTheJuggler]

The process wasn't blind, except for the two that they switched (and even then Shermer wasn't blind to the switch). But why would you expect agreement on readings done for women of different ages?

And even those two were only blind when they reviewed the wrong reading.

To me the whole thing is useless. The "hits" recorded for people who got the right reading are to be compared against what?

If anything, they should be compared to Forer's results, which run about 85%. (That is, on he gets an average of 4.26 on a scale of 0 to 5 for asking people how well they think his one-size fits all reading fits them specifically, which is really about the only measure we're getting from this exercise.) I haven't watched the video in a while, but IIRC the percentage quoted (which is only a subjective measure of how well it suits them and not really an objective measure of hits and misses) was lower than Forer's results.

So the guy isn't even very good at his humbuggery.

And talk about shoehorning!

He said the woman might be a martial artist or a dancer. She was, in fact, neither, but claimed uncanny hits on both because she used to take a class in tae kwan do and she likes to dance. I'll bet if you question women in her age cohort--say +/- 5 years, you'll find very few that have never had some kind of connection to martial arts (at least as near as "once took a class") AND that don't like to dance.

 

[skeen]

I'm certainly unimpressed by this. The amount of information he had was great - using just age and gender, you can formulate a lot of generic statements about someone's life. When you're fine with getting some of them wrong, you can most certainly get a high hit-rate most of the time.

It's no more different than cold reading, except you're not getting as much information, and you're also not feeding back as much information, as is evident in this video.

I also agree that the whole testing process seemed flawed to me; it has already been established many many times that these general readings can apply to most people - the switching of the last two individuals in the end did little to appease that, given that they were of considerable age difference (perhaps 20 years?).

They all should have gotten eachothers' readings, and they all should have been within the same age bracket, with gender taken into consideration also.

 

[Rodney]

Wow I wonder who was lying lol? Either their was a second part or not it seems like there wasn't therefor pat is laying.

I assume that you meant to say "lying" but, in any event, all Shermer says is:

"My memory on what we were trying to do that day of filming is a little vague, but if I recall correctly there was to be another stage of the experiment where Armstrong had to match his astrological readings with the profiles of a group of new subjects, and then have them do the same, picking out their reading from a batch he produced, and then compare them. But we ran out of time."

So, Shermer did not address the allegation that he delivered an "ending so irritable and unbalanced that Jeffrey called the show’s producer and pointed out that it was 'out of integrity' for them to not point out that the experiment had gone completely in favour of astrology."

 

[Rodney]

He said the woman might be a martial artist or a dancer. She was, in fact, neither, but claimed uncanny hits on both because she used to take a class in tae kwan do and she likes to dance. I'll bet if you question women in her age cohort--say +/- 5 years, you'll find very few that have never had some kind of connection to martial arts (at least as near as "once took a class") AND that don't like to dance.

Evidence?

 

[Limbo]

So, Shermer did not address the allegation that he delivered an "ending so irritable and unbalanced that Jeffrey called the show’s producer and pointed out that it was 'out of integrity' for them to not point out that the experiment had gone completely in favour of astrology."

Yes, that's exactly what I would like to see Shermer address.

 

[Rodney]

I'm certainly unimpressed by this. The amount of information he had was great - using just age and gender, you can formulate a lot of generic statements about someone's life. When you're fine with getting some of them wrong, you can most certainly get a high hit-rate most of the time.

It's no more different than cold reading, except you're not getting as much information, and you're also not feeding back as much information, as is evident in this video.

I also agree that the whole testing process seemed flawed to me; it has already been established many many times that these general readings can apply to most people - the switching of the last two individuals in the end did little to appease that, given that they were of considerable age difference (perhaps 20 years?).

They all should have gotten eachothers' readings, and they all should have been within the same age bracket, with gender taken into consideration also.

So why did Shermer agree to this protocol, then?

 

[skeen]

So why did Shermer agree to this protocol, then?

What difference does it make? It doesn't prove anything to anyone. Are you saying that my solution would not have proved one way or another whether there was something to this? Are you saying that the protocol here did achieve that? Given all we know about general statements in astrology, as demonstrated by Randi?

 

[fls]

Evidence?

<raises hand>

Linda

 

[fls]

So why did Shermer agree to this protocol, then?

I was thinking about this. It may be useful to first show the effect under unblinded conditions, so that everyone agrees that the effect is present and obvious. Then demonstrating what happens when everyone is blinded - the loss of certainty, etc. - has a greater impact. The protocol may have called for both parts, but Shermer only had resources to carry out the first part.

Linda

 

[Femke]

Evidence?

<raises hand too>
And this goes as well for many of my friends from college (at least those I ever asked), if a course in Self Defence for Women counts as a martial art.

Femke

 

[Rodney]

I was thinking about this. It may be useful to first show the effect under unblinded conditions, so that everyone agrees that the effect is present and obvious. Then demonstrating what happens when everyone is blinded - the loss of certainty, etc. - has a greater impact.

Except that Armstrong's best results came with respect to the two women whom Shermer gave the wrong analyses.

 

[skeen]

And who just happened to be many years apart. It's unlikely the younger woman would have been married twice, for example.

 

[Rodney]

<raises hand too>
And this goes as well for many of my friends from college (at least those I ever asked), if a course in Self Defence for Women counts as a martial art.

Femke

Armstrong specifically said a "dancer or martial artist" so I don't think merely taking a Self Defence for Women course would count. Now, admittedly, the woman in question is neither a dancer nor a martial artist, but on the other hand, she says that she took a Taekwondo course and likes to dance, so both are at least generally covered. And I seriously doubt if most women her age have taken a martial arts class, Linda and you to the contrary notwithstanding. According to Wikipedia:

"As of 2003, over 1.5 million US citizens practice martial arts." See http://en.wikipedia.org/wiki/Martial_arts

Not exactly a large portion of the 2003 U.S. population of about 300 million, although I would guess perhaps more like 3 million would have taken a martial arts class at some point. However, I would also guess that more men than women have taken martial arts classes. Overall, I would estimate that perhaps 1-2% of that woman's age group in the U.S. have taken a martial arts class.

 

[Rodney]

And who just happened to be many years apart. It's unlikely the younger woman would have been married twice, for example.

You evidently don't know my niece.

 

[skeen]

Armstrong specifically said a "dancer or martial artist" so I don't think merely taking a Self Defence for Women course would count.

Considering that in any kind of magical reading, be it astrology or psychic, the person is looking for something that matches. You can bet your ass she would have made it fit, just as they have made almost everything fit.

There are no specifics in this kind of a reading, only general statements. "You didn't have a good relationship with your father", for example, is maybe a 50/50 chance more or less.

 

[JoeTheJuggler]

Evidence?

Of what? That they counted her saying she once took a class in tae kwan do and that she likes to dance as a hit for the claim that she might be a martial artist or a dancer? That's in the video. It's obviously shoehorning.

My speculation that allowable hits to the statement that she might be a martial artist or a dancer is casting a very wide net is just speculation. It's certainly a lot wider than requiring an actual hit (like actually being a martial artist or a dancer).

 

[JoeTheJuggler]

Except that Armstrong's best results came with respect to the two women whom Shermer gave the wrong analyses.

You can't just pick out his "best" results though. Overall, he didn't do quite as well as the result Forer gets.

 

[JoeTheJuggler]

Armstrong specifically said a "dancer or martial artist" so I don't think merely taking a Self Defence for Women course would count. Now, admittedly, the woman in question is neither a dancer nor a martial artist, but on the other hand, she says that she took a Taekwondo course and likes to dance, so both are at least generally covered.

Yeah--so admittedly it was a double miss that was counted as a double hit. (For that matter, how do you know that what the woman described as a "tae kwan do" class wasn't in fact actually titled "Self Defense for Women"?)

That's the problem with this kind of demonstration. Anything goes.

That's why I'm saying the ONLY way you can evaluate these results is that the subjects' responses (counting hits and misses) was simply their subjective rating of how accurate they think the readings were for them. (Exactly the same question as the Forer experiment asks.) So we just convert both results to a percentage and find that Armstrong scored just a little lower (but virtually the same) as the results of Forer's tests.