• Quick note - the problem with Youtube videos not embedding on the forum appears to have been fixed, thanks to ZiprHead. If you do still see problems let me know.

Micro results in a macro world

If you've got one person who can reliably change the output of a random number generator...does anyone else smell $1 million around here?

But wait...no one has ever succesfully gained that prize...
 
Lazy thinking! Something has an apparently negligible effect so it doesn't matter? Hah!

Paranormal events are a lousy example of this. The law of big numbers allows for statistical flukes. Of course they are bunk.

Why not try the law of unexplained limitations if you want a stick to beat psychics with (eg why can Uri Geller bend cutlery but not crowbars).
 
rwald said:
If you've got one person who can reliably change the output of a random number generator...does anyone else smell $1 million around here?

But wait...no one has ever succesfully gained that prize...


The issue is one of replication. There hasn't been any reliable replication as far as I know.
 
Re: Re: Re: Re: Re: Micro results in a macro world

Juryjone,


----
I must admit I've never read the PEAR tests - I have no background in statistics and therefore the figures are meaningless to me.
----


No offense, but if you've never read the articles about PEAR, why do you feel the need to comment on them:


----
How about starting with the PEAR experiments that were debated ad nauseam here months ago? They found that, once you whip all the numbers together, there could be a slighter greater result than chance that a random number generator could be influenced in a general direction (i.e., more ones than zeros generated or vice versa). Even if the results could be replicated repeatedly, of what significance is this? Is this of any practical use? “Oh my god, they’ve released a psychic who can make our RNGs slightly less random! Run for the hills!” Yet it seems that this is the best study that there is to offer.
----


It seems like you are baseing all of your 'knowledge' about PEAR on what was written in this forum. How is that anywhere near being skeptical on the issue of PEAR?

Shouldn't you read some articles on PEAR or something.

There are many of us in this forum who could explain the statistical issues.
 
Re: Re: Re: Re: Re: Re: Micro results in a macro world

Whodini said:
It seems like you are baseing all of your 'knowledge' about PEAR on what was written in this forum. How is that anywhere near being skeptical on the issue of PEAR?

Shouldn't you read some articles on PEAR or something.

There are many of us in this forum who could explain the statistical issues.

Well, do you think that if I read them that my view on the significance of the results would change? You said yourself, "There is some significance in individual analyses, but when you combine them, you get a very significant overall effect." As a layman in statistical matters, it would seem that if you put together a bunch of results with "some significance", you couldn't possibly get a "very significant" result, just as you can't add up a bunch of test scores of 60 and expect to get a C. I don't know if I'd ever be able to understand a meta-analysis that came up with better results than any single study.

By the way, thanks for getting my name correct. "Juryone"? Even I'm not egotisitcal enough to imagine I'm a jury of one. :)
 
Re: Re: Re: Re: Re: Re: Re: Micro results in a macro world

juryjone said:


As a layman in statistical matters, it would seem that if you put together a bunch of results with "some significance", you couldn't possibly get a "very significant" result, just as you can't add up a bunch of test scores of 60 and expect to get a C. I don't know if I'd ever be able to understand a meta-analysis that came up with better results than any single study.

The word "meta-analysis" should always raise a red flag. Any research that appeals to meta-analysis for significance has to be scrutinized closely.

A basic problem is something called "the file drawer effect."

Imagine that there are 10 placebo controlled studies looking at the effect of drug X on disease Y. Three of those studies show that 70% of patients getting X improve, compared to 50% getting placebo; the rest are more like 50% vs 50%. Maybe even one study shows that people getting drug X do worse than placebo.

In any of the three studies seeming to show a positive effect for Y, there aren't enough patients to prove that the effect is statistically significant. But when the three studies are pooled, the total number of patients is larger and the effect then becomes statistically significant.

But it's not fair retrospectively picking out only the more positive studies for the pool! The seven studies "left in the file drawer" have to be included somehow.
 
Re: Re: Re: Re: Re: Re: Re: Re: Micro results in a macro world

DrBenway,


I agree with your discussion of the File Drawer Effect. Any competent researcher doing a meta-analysis needs to synthesize the results from all studies with a similar measure, not just the ones with good z-scores.
 
Re: Re: Re: Re: Re: Re: Re: Micro results in a macro world

Juryjone,


----
Well, do you think that if I read them that my view on the significance of the results would change?
----


Well, probably not, but, you are basically just arguing about the article, but you've only read the summary but don't know any of the technical details.

I'm definitely not saying you can't get anything out of it, but rather that you could get more out of it if you read it all.

Bigger is better than the smaller
But bigger and smaller is better than the biggest.
So read the entire article in addition to the abstract.


----
As a layman in statistical matters, it would seem that if you put together a bunch of results with "some significance", you couldn't possibly get a "very significant" result, just as you can't add up a bunch of test scores of 60 and expect to get a C.
----


One certainly can do that. Keep in mind that the opposite could also happen. (if one adds up negative z's and positive z's, they could 'cancel')

A z-score of 1 is not significant at the 5% level, but if you use a meta-analysis summary of (for example)

R = (z1+z2+z3+...+zn)/sqrt(n),
(z's are from a standard normal)

if all the zn's are 1, or larger for example, then you get a very significant result at the 5% level.

Which makes sense because individually they don't mean much, but they are all positive 'effects'.
 
The thing is, juryjone's original point was that even if the meta-analysis proved that psychics can reliably make a machine output 5% more 1's than it should, so what? If any one person were talented enough to do something more meaningful, then a meta-analysis wouldn't have been needed. It would require a slightly different way of thinking about things, but it would have no practical effect on people's lives.
 
Rwald,


----
The thing is, juryjone's original point was that even if the meta-analysis proved that psychics can reliably make a machine output 5% more 1's than it should, so what?
----


One couldn't conclude that from a meta-analysis.

But in any case...


----
If any one person were talented enough to do something more meaningful, then a meta-analysis wouldn't have been needed.
----


Understood, but the meta-analysis wouldn't be done with just that one person's results. A meta-analysis would synthesize the results from, say, 100 different people.
 
Unfortunately, the PEAR reports are worthless garbage. I can't say whether the experiments conducted by PEAR have shown something or not. But I have read the reports, and they are worthless garbage.
 
PixyMisa said:
Unfortunately, the PEAR reports are worthless garbage. I can't say whether the experiments conducted by PEAR have shown something or not. But I have read the reports, and they are worthless garbage.


LOL.

Care to elaborate on your skeptical treatment of the subject.

:rolleyes:
 
Whodini said:
Keep in mind that the opposite could also happen. (if one adds up negative z's and positive z's, they could 'cancel')

A z-score of 1 is not significant at the 5% level, but if you use a meta-analysis summary of (for example)

R = (z1+z2+z3+...+zn)/sqrt(n),
(z's are from a standard normal)

if all the zn's are 1, or larger for example, then you get a very significant result at the 5% level.

Which makes sense because individually they don't mean much, but they are all positive 'effects'.

Whew. As I told you, it takes very little in the way of statistics for me to get out of my depth. If you haven't "dumbed it down" as much as you could already, could you explain that in layman's terms?


Originally posted by rwald
The thing is, juryjone's original point was that even if the meta-analysis proved that psychics can reliably make a machine output 5% more 1's than it should, so what? If any one person were talented enough to do something more meaningful, then a meta-analysis wouldn't have been needed. It would require a slightly different way of thinking about things, but it would have no practical effect on people's lives.

Yes, indeed, that is my original point. Thank you.
 
I'll try to explain what Whodini's saying. Let's say you have a bunch of experiments, each of which shows a small (but statistically insignificant) correlation. Individually, each experiment doesn't mean anything (it could be due to random chance). But if all those different experiments all had a positive correlation, than maybe there really is a positive correlation. That's the gist of it.
 
Juryjone,


----
Whew. As I told you, it takes very little in the way of statistics for me to get out of my depth. If you haven't "dumbed it down" as much as you could already, could you explain that in layman's terms?
----


A z-score is something like z = (x-ave of x's)/standard deviation of the x's. It is basically a 'standardized score'. -just a score put on another scale, like Farenheit to Kelvin.

The idea is that while x may have been on a lbs scale, or a height scale, or on a temperature scale, x may have been on another scale in a different analysis, so the scores are standardized so we can compare scores.

Based on the bell curve, a z greater than about 2 (or less than about -2, because the curve is symmetric) is deemed 'significant' at the 5% level.
The z's can take on any value from, say, -10 to 10, where the more positive or more negative the number is, the more 'significant' the result is. Most z's take on values between -3 and 3.

"At a 5% signifigance level' means that if there is no paranormal stuff going on, we'd expect a z this large only 5% of the time or less. So if we get a very very large z, this is evidence against 'no paranormal stuff going on'.

A meta-analysis combines these z's from many experiments. (There are conditions: like they have to be similar experiments where things were measured the same or similar way)

A typical meta-analysis combination of the z's is:

R = squareroot(n)*average of z's,

where n is the number of z's you are combining.

(which is mathematically the same as (z1+z2+z3+...+zn)/sqrt(n), which is what I wrote in the previous message)

We'd be interested in seeing if this R is greater than 2 or less than -2. If it was, the results would be 'significant'.

A z-score of 1 is not significant at the 5% level. But, if all of the z's are mostly of the same sign (+ or -) and are combined, the result is that R could be significant.

Which makes sense because individually they don't mean much, but they are all positive 'effects'.
 
Whodini said:
LOL.

Care to elaborate on your skeptical treatment of the subject.
I've got you on "ignore", but I expected this sort of response.

Start here.

Someone here posted a further analysis of the PEAR papers addressing some points that I missed. I'm sure you could find it if you ever bothered to look.
 
PixyMisa,


You have me on ignore? Uh, ok...


Your 'analysis' amounts to name calling.

The fact that you can't wrap your brain around the abstract is moot.

The fact that a test wasn't blinded doesn't make the science apriori bad.

The fact that the experimenters can dictate the experiment (encourage a playful mood) is not bad. That is what experimenters do, they set the parameters of the experiment.

You said:

----
Not keeping records also being a popular tool of experimental science.
----


But they did NOT say they didn't keep records. They said that they did not "maintain systematic records on the relative effectiveness of the various personal strategies developed by the participants in their approach to the task, or on any of their psychological characteristics."

That is hardly the same thing as saying no records were kept, period.

For example, you completely ignore, in your review, the sentences RIGHT AFTER all of that where they do talk about some of the things they recorded. You also ignore all the tables and graphs they had in the paper. Not to mention all their appendices. You think that these magically appeared without them keeping some type of records?

They also have several sections where their p-values were NOT significant. So so much for the 'only publishing the hits' theory.

Can you explain the fact that the 653-trial formal database had a composite z-score of 5.418 (pvalue < 3x10^ -8 )? And I'll give you a hint, saying "meta-analysis" as your answer doesn't explain anything.
 
Re: Re: Micro results in a macro world

Interesting Ian said:

...we had better start addressing what the ultimate nature of the self is and whether there is a life after death etc etc.

There is no life after death. That subject has now been addressed.
 

Back
Top Bottom