The "Process" of John Edward

[Lurker]

From http://www.math.uiuc.edu/~hildebr/361/feb14.html

Math 361 at Univ Illinois

"I introduced two important approximations to the binomial distribution, the Poisson approximation and the normal approximation. The Poisson approximation is best suited in the case of rare events and a large number of trials. while the normal approximation works well when the number of trials is More precisely, in order for the Poisson approximation to be accurate, p should be very small, n should be large (ideally of order 1/n), and k should be small (which it almost always is, since the cases of interest are those when k is 0, 1, 2, or some other very small number). If these conditions are not satisfied, the Poisson approximation should not be applied, and it may be completely off."


Bill, what was I questioning about number of trials, probability all effecting the accuracy of Poisson?

Lurker

 

[Thanz]

Mr. Hoyt -

You have done a whole lot of crowing in this thread, but you have still neglected to answer my question which I believe goes to the heart of the validity of your analysis. Here it is again, in case you missed it:

I would like to know why you think it is appropriate to count "And they're also talking about somebody who would be known as either Richard or Rich, because a big R-connection that comes up connected to you" as more than one guess. In my count, I counted this as one R guess. From your description, I gather you counted it as 3 R guesses.

 

[Lurker]

Thanz:

Bill sure is a good dancer. It is hard to keep hopping about refusing to answer reasonable questions but he seems to have mastered it.

Lurker

 

[BillHoyt]

From http://www.math.uiuc.edu/~hildebr/361/feb14.html

Math 361 at Univ Illinois

"I introduced two important approximations to the binomial distribution, the Poisson approximation and the normal approximation. The Poisson approximation is best suited in the case of rare events and a large number of trials. while the normal approximation works well when the number of trials is More precisely, in order for the Poisson approximation to be accurate, p should be very small, n should be large (ideally of order 1/n), and k should be small (which it almost always is, since the cases of interest are those when k is 0, 1, 2, or some other very small number). If these conditions are not satisfied, the Poisson approximation should not be applied, and it may be completely off."


Bill, what was I questioning about number of trials, probability all effecting the accuracy of Poisson?

Lurker

You really don't understand that paragraph, do you? Go back over it. The Normal distribution is also inaccurate? Huh?

The discussion there, sir, is about using Poisson to approximate binomial. It does not say Poisson is a lesser distribution any more than it says the Normal is a lesser distribution because of the inaccuracies when the normal is used to approximate binomial!

Wow! We have been through this before. Poisson is not "binomial lite". Normal is not "binomial lite". These sources discuss how accurately Poisson and Normal model Binomial.

Now you get my monkey...

Please stop. You are merely demonstrating how militant you can be in your ignorance. Even the monkey is shaking his head.

 

[Instig8R]

Concerning anything substantive? No. The only thing that I conceded was that it appeared JE might have been wrong in a part of his interpretation about why the "Malibu Shrimp" recipe was kept "secret".

He interpreted it as meaning that it was secretly based upon one of Deborah's mother's recipes, which he believed was what he was being shown him by her mom, and Deborah said the real reason it was kept secret was because of the questionable clams that she and her friend used when making it, although she did admit that the recipe was based, at least loosely, upon her mom's recipe.

Instig8R feels JE pressured Deborah into admitting that fact, and considers this is a monumental matter. I feel that it could indeed be true, that the recipe was loosely based on one of Deborah's mother's recipes, since most daughters do tend to learn some cooking from their moms.

In any case, I also feel that this whole matter was blown way out of proportion by Instig8R, since the balance of that reading stands, and it was a good reading, with a lot of excellent, accurate hits, and in no way does any of that hinge upon this one, rather irrelevant point.......neo

Hey, neo-- Perhaps the following will help jog your memory... It is our dialogue on 6/25/02 (before you saw the edited reading on television):

(Instig8R)
"Now, I wonder why he didn't ask in what way the sitter's mom contributed to that recipe..."

(neofight):
"Why is that even important? Basically it's been said already. I think it's clear from the context that the woman and her friend simply took one of her mother's recipes that she probably really liked, and improved upon it, adding various ingredients and truly making it a culinary delight. They were probably very proud of the results, and were loath to admit it was not entirely theirs. I would be surprised if there was anything more to the story than that, but it would be nice to find that out for sure....neo"

You seem to have changed your opinion since then. You argued quite vigorously as to the meaning of that hit in June 2002. The "hit" sure seemed relevant in June, 2002. Funny how viewing the edited reading affected your memory and your interpretation of the live reading, isn't it? Memory is a funny thing, isn't it?

The point is that JE said that he had Deborah's friend (Helen) and Deborah's mother, together. Mom showed up, specifically to call attention to that stolen recipe. Remember? The recipe was based on something that Mom made, and she never got the credit that she deserved. The point is that JE pressured Deborah into a false validation. She never stole a recipe from her mother, and all those false accusations were edited out of the final reading.

You were so sure, in June 2002, that Deborah would NOT have acknowledged a lie, and you seemed very impressed that JE was let in on that little secret. Yeah, Deborah was busted by the spirits of her mom and her friend. If not for editing, this reading would not have been fit for broadcast.

 

[Lurker]

Wow! We have been through this before. Poisson is not "binomial lite". Normal is not "binomial lite". These sources discuss how accurately Poisson and Normal model Binomial.

Now you get my monkey...
[img[http://www.baltobluegrass.com/bbggraph/monkey.gif[/img]

Please stop. You are merely demonstrating how militant you can be in your ignorance. Even the monkey is shaking his head.

Have you derived the Poisson Distribution, Bill? I could be wrong but isn't it derived by taking the limit of the Binomial Distribution as n=>infinity? What implication does this have?

Lurker

 

[CFLarsen]

The point is that JE pressured Deborah into a false validation.
...
If not for editing, this reading would not have been fit for broadcast.

Two points, impossible to argue against.

 

[TLN]

Two points, impossible to argue against.

We'll see...

 

[Walter Wayne]

Bill,

I think I see what you are getting at. Based on 1 trial, we expect JE to guess J 0.1336 times. If on average over our tests JE guesses J 0.25 times/trial then the p value will be identical.

You are normalizing the units of the average and sample. Instead of comparing the expected 0.1336 J's/trial compared to found 25 J's in 100 trials, you are comparing expected 0.1336 J's/trial to 0.25 J's/trial.

Walt

Edit: My post has some redunancies and repeats itself.

 

[T'ai Chi]

Hey - I am interested in other letters!

Sorry Thanz!!

I should have specified that only Bill is not interested in analyzing the other high frequency letters (at least according to his analysis that only focuses on the high frequency letter J).

 

[Walter Wayne]

Sorry Thanz!!

I should have specified that only Bill is not interested in analyzing the other high frequency letters (at least according to his analysis that only focuses on the high frequency letter J).

Actually, somewhere in this thread he mentions that one could also check the theory by seeing if low frequency letters were under represented. I'd quote it, but it is a long thread.

Walt

 

[T'ai Chi]

I agree with that WW.

My point is that Bill's analyses only consider one letter at a time. He knows that doing the same test multiple times for the letters of interest introduces more error than doing one test that tests several letters at once. If he was interested in testing multiple letters, I'd think he'd find a more appropriate analysis.

 

[69dodge]

Originally posted by BillHoyt
The discussion there, sir, is about using Poisson to approximate binomial.

Certainly.

It is my impression that the discussion here is about the same thing.

 

[69dodge]

Originally posted by Walter Wayne
I think I see what you are getting at. Based on 1 trial, we expect JE to guess J 0.1336 times. If on average over our tests JE guesses J 0.25 times/trial then the p value will be identical.

You are normalizing the units of the average and sample. Instead of comparing the expected 0.1336 J's/trial compared to found 25 J's in 100 trials, you are comparing expected 0.1336 J's/trial to 0.25 J's/trial.

Yes.

The sum of a bunch of independent random variables, all of which have Poisson distributions, itself has a Poisson distribution. The mean of the sum is the sum of the means, which don't have to be identical, though here they are.

Edited to add: I'm not sure what I was thinking when I wrote the above. It's correct, but it's got nothing to do with what you wrote.

I guess I don't quite understand what you wrote, actually. Can you elaborate?

 

[69dodge]

Originally posted by BillHoyt
Poisson doesn't care about n. If we look at an acre of land and find 12 dead crows, what was the N? Who knows, who cares?

The n is the number of crows in the world that could possibly have ended up dead on that acre of land. And p is the probability that any given one of them actually would do so.

We don't care about the exact value of n, but only because we do know that it is much larger than 12.

Poisson is not parameterized on N.

Quite true. However, if we're considering using a Poisson distribution with parameter np to approximate a binomial distribution with parameters n and p, we should ensure that n is sufficiently large and p is sufficiently small. Otherwise, the approximation will be very poor.

In all the cases that I know about where a Poisson distribution is used, the theoretical justification is that it is an approximation of an underlying binomial distribution with large n and small p. Does anyone know of a different reason why we should expect a physical system to have a Poisson distribution?

 

[69dodge]

Originally posted by BillHoyt
Now let's go to counting initials. We pick an initial that has a frequency of .5. We count 9 such initials in a field of 10. We use a mean of 5 and look at the cdf for >= 9. And we get .03.

Based on your figure of 0.03, I believe you meant '>', not '>='. I will use '>' in the rest of my reply.

I'm not sure what you are claiming. You give an answer of 0.03. But what is the question? I see two possibilities.

1. Let X be a random variable with a Poisson distribution whose mean is 5. What is the probability that X > 9 ?
   
   The answer to this question is 0.0318.
   
2. I pick ten letters at random. Each has, independently, a probability of 0.5 of being a J. Let Y be the number of J's that I pick. What is the probability that Y > 9 ?
   
   The answer to this question is 0.000977.
   

That the answers differ can mean only one thing: Y does not have a Poisson distribution with mean 5. In fact, it has a binomial distribution with n = 10, p = 0.5.

Now let's pick an initial that has a frequency of .05. We count 9 such initials in a field of 100. We use a mean of 5 and look at the cdf for >= 9. We get .03.

Again, I see two possibilities for what you are asking.

1. same question as before.
   
   same answer as before.
   
2. I pick 100 letters at random. Each has, independently, a probability of 0.05 of being a J. Let Z be the number of J's that I pick. What is the probability that Z > 9 ?
   
   The answer to this question is 0.0282.
   

Again, the two answers differ. And again, this is because Z does not have a Poisson distribution with mean 5. It has a binomial distribution with n = 100, p = 0.05.

However, the two answers are closer than they were before. This is an illustration of the fact that, as n increases and p decreases, with np remaining constant, the binomial distribution with parameters n and p approaches the Poisson distribution with mean np.

 

[Lurker]

The n is the number of crows in the world that could possibly have ended up dead on that acre of land. And p is the probability that any given one of them actually would do so.

We don't care about the exact value of n, but only because we do know that it is much larger than 12.Quite true. However, if we're considering using a Poisson distribution with parameter np to approximate a binomial distribution with parameters n and p, we should ensure that n is sufficiently large and p is sufficiently small. Otherwise, the approximation will be very poor.

In all the cases that I know about where a Poisson distribution is used, the theoretical justification is that it is an approximation of an underlying binomial distribution with large n and small p. Does anyone know of a different reason why we should expect a physical system to have a Poisson distribution?

You write much clearer than I did but that is what I was trying to get across in my posts which a certain other poster continually argued around.

Lurker

 

[Thanz]

Sorry Thanz!!

I should have specified that only Bill is not interested in analyzing the other high frequency letters (at least according to his analysis that only focuses on the high frequency letter J).

That could be because we don't have enough data to do a more comprehensive test on multiple letters. Of course, Mr.Hoyt won't admit this, and will stick to his flawed J analysis, as it may mean admitting that I was correct when I said the sample size was too small.

 

[BillHoyt]

In all the cases that I know about where a Poisson distribution is used, the theoretical justification is that it is an approximation of an underlying binomial distribution with large n and small p. Does anyone know of a different reason why we should expect a physical system to have a Poisson distribution?

We are not trying to approximate binomial with Poisson here. We are simply using Poisson. Period. Yes, radioactive decay is a physical system that has a Poisson distribution.

 

[Thanz]

We are not trying to approximate binomial with Poisson here. We are simply using Poisson. Period.

Why? What makes Poisson the appropriate tool here?