carlson test and debunking randi

Humans have more in common than they like to think, which i believe makes astrological interpretation subject to all kinds of comments.


This is one of the things that makes astrology appear to work. If all people actually have a lot in common, then if an astrologer produces a reading for an individual customer, the customer is likely to recognize themselves in it as long as it is recognizable by anyone (see also confirmation bias). This is probably why it falls down if people are asked to identify their own reading from a set of other readings: they recognize themselves to the same degree in all of the readings.

I think Randi has used an illustration of this (although with a brief horoscope rather than a full reading): giving each member of a school class a horoscope and asking if they think it is accurate. They invariably all do, and he then reveals that all the horoscopes are the same.
 
Last edited:
Maybe the current tests are not the ideal i dont know.
What was wrong with the test? (and why did this doubt not surface before the results were announced?)

Having said that i m reading a western astrology book in German by a german.
The good thing about German Ernsthaft is that they never write crap.
It is the most accurate book i ever came across, and it was meant only to help me with my german.
How do you objectively quantify this "accuracy"? How does it differ from pattern matching?

So i want to test western astrology and especially Astrocartography, which shows you the best places in thevworld for just about anything.
IE: S. Stallone has Jupiter in MC in New York, where he made movies where he has a good guy vibe, and when he moved to Hollywood where he has Mars inthe same position, he made rather marcial movies and played rather martial characters like Rambo,Rocky...
I thought you had less faith in Western Astrology as it has "lost its root due to persecution"? Why move on to something you have less confidence in when your favoured approach (Chinese Astrology) has done so poorly so far? Wouldn't it be better to just analyse what went wrong in this test rather than throw in more noise?

Anyway, I think "I.E." above should have been "E.G." otherwise you are claiming this only works for Sylvester ... which might also be true. Could this particular example not just be because the box office results for the first "marcial" film was very good so he just carried on making them while they were in vogue?

I'd like to sum up how this has gone so far ...
  • My position: Astrology doesn't work unless it can be proven to in reasonable tests.
  • Your position appears to be something more like ... some form of astrology works, but you have yet to find it. (You claim to have found it until a test shows it to be something more like ego-massaging, broad terms/vagueness, or pattern matching and guessing.) Is that true?
 
today is an 8 and i got bad news about the test...:)
... and I'm sure that lots of people "without an 8" got much worse news than you today ... and some people "with an 8" probably had a really mundane day ... very interesting altogether ...

In any case, wouldn't the day you did the readings be more important than the day you got the results? And what about the day that people checked their charts ... maybe they all had "bad days" due to their numbers?

I note that the number of tests ... "7" ... is the same as your important age number (34 ... 3+4=7) ... did numerology have a bad influence here on the test? Should the tests have used a number of chart which was not significant to you?

Do the numbers make things happen, or do you make the numbers happen? Do the numbers happen anyway and you just notice the ones you want to?
 
Last edited:
i would like the participants to tell where i got it wrong and where i was vague so i can check my reading skills and know what they expect from astrology:)

Claims about the test made after the fact, "Maybe the current tests are not the ideal...." and shifting the blame to the participants, ie. “what they expect from astrology” are intellectually dishonest.

You were expected to prove your claim that astrology "tells your personality in detail and possibly the kind of situations youll get in", based on the testing protocol you agreed to, you didn’t.

Typical "woo woo" excuses.
 
What was wrong with the test? (and why did this doubt not surface before the results were announced?)
There are actually a number of things wrong with the test which were discussed, but it was agreed to go ahead with the test anyway.

Primary among the problems is that of "sensory leakage". Even a cursory examination of this thread would show idunno a fairly small potential pool of candidates each who has posted things that might reveal something about themselves. But this mostly favors the chart-maker, so it can be pretty much discarded.

Another big problem is that the sample is so small that it might mask a weak effect. However, idunno seemed to indicate that astrology/numerology was not a weak effect so that problem is mitigated somewhat.

The biggest problem with the test, in my opinion, is a bit like reverse confirmation bias, let's call it "negation bias". This is sort of the opposite effect that was seen by Randi and by Derrin Brown when they handed the same reading to every member of a class, telling them it was their "personal" reading and 80-90% agreed it was accurate.

Negation bias could occur because the pool of candidates mostly share the trait of being skeptics. As such, they are not likely to be neutral in their beliefs on astrology or numerology, but rather will tend not to believe in either. While I don't think that any of the candidates would deliberately prejudice the results by picking the reading that was most unlike them, it is possible that they are less likely than the public in general to subjectively agree that a chart matches them, even if, objectively it does. Note that several of the targets indicated that none of the charts matched them. Ideally a pool of potential targets should have no knowledge and no opinions of astrology to avoid confirmation bias and negation bias.

Still, assuming that everyone was honest, there should have been some effect if astrology has any factual basis. Either this effect does not exist or it was masked by small sample size. My experience with astrology in college suggests it is the former.

I commend idunno for taking this test and for fairly stating that he "must accept the negative results." I must immediately then chide him for trying to rationalize the negative results by stating:
idunno said:
Humans have more in common than they like to think, which i believe makes astrological interpretation subject to all kinds of comments.
While it is true that the test is not ideal, the proper scientific way to deal with this is to redesign the test without seeking a specific outcome. Idunno should be equally willing to accept the outcome that "astrology doesn't work" if that is what the tests reveal, as he is to mine the data for positive results.
 
Having said that i m reading a western astrology book in German by a german.
The good thing about German Ernsthaft is that they never write crap.
It is the most accurate book i ever came across, and it was meant only to help me with my german.
So i want to test western astrology ...

You've got the same seven birthdates, and I will be happy to scrub and redistribute, if you want to do western-style profiles.
 
You've got the same seven birthdates, and I will be happy to scrub and redistribute, if you want to do western-style profiles.

As idunno has claimed the current testing protocol is not "ideal", I'd suggest that HE outline a testing protocol.

I'd also suggest that a new test group be chosen.
 
As idunno has claimed the current testing protocol is not "ideal", I'd suggest that HE outline a testing protocol.

Well, we tried this earlier and progress was (ahem) limited.

Part of the problem, I suspect, is that I doubt an "ideal" protocol (from the POV of the astrologer) would not permit simple yes/no right/wrong grading. There's more wiggle room if you use something like a Likert scale and ask people to judge "how accurate" a given reading is. As Forer, Randi, and Brown have all shown, people tend to over-read themselves into a single well-written profile.

I'd also suggest that a new test group be chosen.

Same problem as before -- any group from this forum will still be comprised mostly of skeptics.
 
Last edited:
Also, out of interest (and since the test is over) can we see the un-scrubbed readings drkitten?

In a bit. I want to give the players at home a chance to phone in.

"Caller #5, you're on the air...."
 
I can't help noticing that despite the fact gender was specifically excluded, 4 out of the 7 use the words "he" and/or "his". Fortunately this doesn't invalidate the test because this actually biases the test in Idunno's favour, since it increases the chance of people guessing correctly. For example, any girls had a 1/3 chance of guessing right rather than 1/7.

The combination of gender and age clues almost dictate the matches. For instance, #4 is for a male between the ages of 25 and 44. Except that the future and past tense could be misrepresented just as gender was.
 
The combination of gender and age clues almost dictate the matches. For instance, #4 is for a male between the ages of 25 and 44. Except that the future and past tense could be misrepresented just as gender was.

Er, "bring it on." Challenges like this are part of the reason that I'm withholding the actual data for a few days. If you would like to post a set of suggested matchings between dates and profiles, I will comment publically upon it.
 
Drkitten – Agreed with the previous lack of progress/success in getting idunno to outline a testing protocol. Though…it’s not a one horse race; that is to say there may be other astrologers that would be willing to participate.

While I congratulate idunno for his willingness and participation, his limited time availability as well as lack of experience as an actual astrologer makes him a less then ideal candidate.

Also agreed wrt the test group, though that was not the reason for picking new participants. It had more to do with eliminating any “cross contamination”/”previous knowledge”.
 
Also agreed wrt the test group, though that was not the reason for picking new participants. It had more to do with eliminating any “cross contamination”/”previous knowledge”.

I think the amount of "knowledge" contained in the astrological profiles was sufficiently small that I'm willing to run the risk of cross-contamination. That probably sounded somewhat snide. It was intended to.
 
Drkitten – Agreed with the previous lack of progress/success in getting idunno to outline a testing protocol. Though…it’s not a one horse race; that is to say there may be other astrologers that would be willing to participate.

While I congratulate idunno for his willingness and participation, his limited time availability as well as lack of experience as an actual astrologer makes him a less then ideal candidate.

Also agreed wrt the test group, though that was not the reason for picking new participants. It had more to do with eliminating any “cross contamination”/”previous knowledge”.

unfortunately i cant convince experienced astrologers to do tests. If they fail people will know and their reputation as good astrologers will be tarnished.
I myself consider myself experienced in wstern astrology, although i had few clients. My tests are mostly on family members and friends.
And yes, i cannot fully trust a skeptic`s feedback on astrology as well as the one from believers. Both can have vested interests.
Still, i do the tests cause I dont really care, as i dont need astrology for a living. But I admit i would like it to work, not because it means there is a God out there, cause i dont need to believe in God, but just for the sake of choosing the right path in life and avoid mistakes in life.
The W. Astrology test will take longer cause I need to have the charts with me, and there is no printer her in the Café.;)
 
Drkitten – Agreed with the previous lack of progress/success in getting idunno to outline a testing protocol. Though…it’s not a one horse race; that is to say there may be other astrologers that would be willing to participate.

While I congratulate idunno for his willingness and participation, his limited time availability as well as lack of experience as an actual astrologer makes him a less then ideal candidate.

Also agreed wrt the test group, though that was not the reason for picking new participants. It had more to do with eliminating any “cross contamination”/”previous knowledge”.

Im not sure what tests would be the right ones given the nature of astrology.
I dont trust Yes or No type of test questions anyway.:) :boxedin:
 
There are actually a number of things wrong with the test which were discussed, but it was agreed to go ahead with the test anyway.

Primary among the problems is that of "sensory leakage". Even a cursory examination of this thread would show idunno a fairly small potential pool of candidates each who has posted things that might reveal something about themselves. But this mostly favors the chart-maker, so it can be pretty much discarded.

Another big problem is that the sample is so small that it might mask a weak effect. However, idunno seemed to indicate that astrology/numerology was not a weak effect so that problem is mitigated somewhat.

The biggest problem with the test, in my opinion, is a bit like reverse confirmation bias, let's call it "negation bias". This is sort of the opposite effect that was seen by Randi and by Derrin Brown when they handed the same reading to every member of a class, telling them it was their "personal" reading and 80-90% agreed it was accurate.

Negation bias could occur because the pool of candidates mostly share the trait of being skeptics. As such, they are not likely to be neutral in their beliefs on astrology or numerology, but rather will tend not to believe in either. While I don't think that any of the candidates would deliberately prejudice the results by picking the reading that was most unlike them, it is possible that they are less likely than the public in general to subjectively agree that a chart matches them, even if, objectively it does. Note that several of the targets indicated that none of the charts matched them. Ideally a pool of potential targets should have no knowledge and no opinions of astrology to avoid confirmation bias and negation bias.

Still, assuming that everyone was honest, there should have been some effect if astrology has any factual basis. Either this effect does not exist or it was masked by small sample size. My experience with astrology in college suggests it is the former.

I commend idunno for taking this test and for fairly stating that he "must accept the negative results." I must immediately then chide him for trying to rationalize the negative results by stating: While it is true that the test is not ideal, the proper scientific way to deal with this is to redesign the test without seeking a specific outcome. Idunno should be equally willing to accept the outcome that "astrology doesn't work" if that is what the tests reveal, as he is to mine the data for positive results.

Yes, usually believers say « Wow ,how accurate. I cant believe it»
Skeptics say.« All wrong mate»
:boxedin:
 
drkitten - Merely a observation and suggestion; no need for snidness.

idunno - If you are experienced in “wstern” astrology, why didn’t you choose that path from the start?
 
Still, i do the tests cause I dont really care, as i dont need astrology for a living. But I admit i would like it to work, not because it means there is a God out there, cause i dont need to believe in God, but just for the sake of choosing the right path in life and avoid mistakes in life.
If you can't prove that astrology works, but you would like it to, and you base decisions on "the right path in life" on astrological "predictions" - then your decisions will be flawed and you will make mistakes in any case. Sometimes you will take the "right" path, and sometimes the "wrong" path - but astrology won't have helped you either way - it just sounds like an excuse to absolve yourself of any culpability in the decision making process.

One day, you will realize that mistakes - like accidents - just happen, and you should learn from your mistakes ...

You don't "need" astrology - you just need to filter the crap information out from all the information available, and make an informed judgement using the good information. Choose your own path accordingly - don't just look for someone or something to tell you what to do all the time.

I think you could paraphrase all that as ... "grow up, you are responsible for your own life".
 

Back
Top Bottom