Thanks Blutoski and Cat.
If I've understood correctly p<=.05 is considered as a threshold for something other than noise in this field of study? If we end up doing only one trial of 20 guesses, what would you set as a pass mark?
I do most of my work to prepare for legal testimony, and the law uses 1 in 20, or p<=0.05.
I would consider p<0.1 (or 1 in 10) to be 'weakly significant', to the point that further research would be needed. 14 of 20 here would be enough to warrant scheduling another trial - I would recommend that the test be repeated, with protocols determining whether that would be done immediately or on another day.
p<0.05 (1 in 20) to me is not 'proof', but it is 'significant'. Without being informed about JREF Challenge Protocol, I would say this might be a step in qualifying. Someone more informed should weigh in here....
p<0.01 to 0.001 (1 in 100, through 1 in 1000). There is a high probability that the results are not due to chance. These are strong results. Something is most likely going on here. I would use the phrase 'highly significant', and maybe even consider the word 'proof', at least tentatively.
Challengers will complain about retests, significance standards, and the like, but don't forget that the claims are of the sort that challenge the very fabric of our knowledge of how our universe works. Big claims require high standards of proof - and in truth, your subject's goal should be get results that would only occur by chance one time in billions, even trillions. This isn't a very long process: 80 or better out of 100 is about a one-in-a billion chance event, as is 23 in a row.
It's also important to stress that replicating results is also a key to science. If your subject truly has the power claimed, then failed re-tests under different circumstances provide extra insight into the nature of the power (doesn't work in direct sunlight, or near large numbers of people, etc. etc.)