davidsmith73
Graduate Poster
- Joined
- Jul 25, 2001
- Messages
- 1,697
Zep, I appreciate the time you've put into trying to find support for your argument, but posting quotes without any explanation as to how they support what you are saying is not very helpful. I'll address the quotes as follows.
This refers to the Chicago trials that were assessed for inter-judge variability. They used rank judging rather than the more sophisticated analysis they developed later. The overall significant result they reported did not include these human rank-judging analyses.
As the quote says, they developed their more objective scoring technique after they found this inter-judge variability mentioned before. The overall positive result they reported only included trials run through this standardised method.
Where's the problem here? The above quote refers to an initial set of 35 trials whereby the participants answered binary descriptor questions. They were highly significant but not as significant as the free response data that was then encoded into binary. This does not imply that the drop in statistical significance was anything to do with the analysis because both used the binary method, nor does it bear any relevence to your claim that PEAR reported they got "no results in the end".
I'm not sure what this quote is saying, precisely, but I guess it is referring to one or more methodological problems we agree were present in the PEAR work. But I fail to see how this supports your claim that PEAR reported they got "nothing in the end".
Again, there's no problem here. This bears no relevance to your claim that PEAR reported no overall result, or your hypothesis that the drop in results were due to the analysis method. Here, they are describing the limitation that binary descriptor questions impose on the participants in terms of conveying a rich representation of their conscious impressions.
Zep, we have agreed that this is a methodological error in the PEAR work. My objection in my last post to you was aimed specifically at your claim that PEAR reported no overall result and your hypothesis that the drop in results were due to more objective analysis, both of which are simply not true. This quote above bears no relevance those claims of yours.
Yes, there was a diminished effect size. Was this down to the analysis method? No it wasn't, as I've explained. Did this diminished effect size mean that PEAR reported overall null results? No it didn't, because they cleary report a significant overall result. You can see it in the abstract. You keep referring to a subset of experiments that got insignificant results and then conclude that this means PEAR reported overall null results, which is absurd.
Here is one such subset of results. What are trying to say by posting this quote? We know that the FIDO data got marginally significant results. The drop in results was not due to the analysis method because they treated this data as binary and got no artifactual positive result. I keep explaining this point to you, but you seem to ignore it or do not understand.
As PEAR clearly state here, we have the FIDO data treated in a number of different ways, including binary, and the p-values are all similar. This shows that the drop in results is not due to the analysis method. Its odd that you post a quote that directly contradicts your hypothesis that the drop in results is due to the analysis method.
The results of the distributive trials were at chance level. This is a subset of results, not the overall results. PEAR reported overall significant results for their entire database (excluding the human judged Chicago trials). Also, these trials that used distributive descriptor questions, when analysed as binary answers, did not get positive results.
To summarise,
PEAR reported an overall highly significant result, contrary to what you claim.
Your hypothesis that the drop in results are due to the analysis method is not supported by the PEAR data.
Ready? OK, here we go. Quotes only from the original PEAR report.
"Although approximately half of these trials demonstrated a strong consistency in the ranks assigned by both the primary and secondary judges and confirmed the acquisition of significant extra-chance information, the others received a wide range of ranks, suggesting that the matches originally assigned to these trials had most likely been arbitrary."
This refers to the Chicago trials that were assessed for inter-judge variability. They used rank judging rather than the more sophisticated analysis they developed later. The overall significant result they reported did not include these human rank-judging analyses.
"Beyond the accumulation of new empirical data, the first major thrust of the embryonic PEAR program was an attempt to alleviate some of these shortcomings by developing standardized methods of quantifying the information content of the free-response data via a series of computer algorithms."
As the quote says, they developed their more objective scoring technique after they found this inter-judge variability mentioned before. The overall positive result they reported only included trials run through this standardised method.
"Although the statistical results of these new trials were not as strong as those of the ex post facto–encoded data, they were still highly significant."
Where's the problem here? The above quote refers to an initial set of 35 trials whereby the participants answered binary descriptor questions. They were highly significant but not as significant as the free response data that was then encoded into binary. This does not imply that the drop in statistical significance was anything to do with the analysis because both used the binary method, nor does it bear any relevence to your claim that PEAR reported they got "no results in the end".
"Even the null results of the 52 exploratory trials are informative in their indication that the features violated in these excursions from the standard protocol, i.e., the percipients’ knowledge of the agent or of the time of target visitation, may be requisites to generation of the anomalous effect."
I'm not sure what this quote is saying, precisely, but I guess it is referring to one or more methodological problems we agree were present in the PEAR work. But I fail to see how this supports your claim that PEAR reported they got "nothing in the end".
"On the other hand, the analytical judging process introduced certain imperfections of its own. For example, the forced ‘‘yes’’ or ‘‘no’’ responses were limited in their ability to capture the overall ambience or context of a scene, or nuances of subjective or symbolic information that might be detected by human judges. Furthermore, while restricting the extracted information to the 30 specified binary descriptors minimized the reporting task for the participants, it precluded utilization of other potentially relevant features in the transcripts, such as specific colors, textures, architectures, or any other details not covered by the questions."
Again, there's no problem here. This bears no relevance to your claim that PEAR reported no overall result, or your hypothesis that the drop in results were due to the analysis method. Here, they are describing the limitation that binary descriptor questions impose on the participants in terms of conveying a rich representation of their conscious impressions.
"Given the less formal nature of the target selection process in the volitional trials, it was possible that the agent’s knowledge of the percipient’s personal preferences or target response patterns could have influenced the target selection and representation, thereby introducing an undue bias into the volitional trial scores. "
Zep, we have agreed that this is a methodological error in the PEAR work. My objection in my last post to you was aimed specifically at your claim that PEAR reported no overall result and your hypothesis that the drop in results were due to more objective analysis, both of which are simply not true. This quote above bears no relevance those claims of yours.
"Notwithstanding, the diminished effect size prompted a new phase of investigation with the goal of achieving a better understanding of the cause of this attenuation and recovering the stronger yields obtained in the original experiments."
Yes, there was a diminished effect size. Was this down to the analysis method? No it wasn't, as I've explained. Did this diminished effect size mean that PEAR reported overall null results? No it didn't, because they cleary report a significant overall result. You can see it in the abstract. You keep referring to a subset of experiments that got insignificant results and then conclude that this means PEAR reported overall null results, which is absurd.
"The composite z-score thus calculated for the 167 FIDO trials was 1.735, indicating a marginally significant overall achievement, but one that was reduced even further from the high yield of the previous data. "
Here is one such subset of results. What are trying to say by posting this quote? We know that the FIDO data got marginally significant results. The drop in results was not due to the analysis method because they treated this data as binary and got no artifactual positive result. I keep explaining this point to you, but you seem to ignore it or do not understand.
"Other than the binary-reduction version, which produced nearly as many extra-chance ‘‘misses’’ as ‘‘hits,’’ the results from the other five methods all displayed relatively close concurrence, marginally significant composite z-scores, and effect sizes only about half that of the ab initio trials and only about a fifth as large as that of the ex post facto subset. Although the proportions of trials with positive scores were above 50% in all the calculations, neither these nor the numbers of significant trials exceeded chance expectation. Clearly, FIDO had not achieved its goal of enhancing the PRP yield, despite its potential sensitivity to subtle or ambiguous informational nuances in the data. Despite some variability among the z-scores calculated for individual trials by the different scoring methods, the general consistency across most of the scoring methods for the composite database suggested that the decreased yield was not directly due to inadequacies in the FIDO scoring algorithms, per se, but to a more generic suppression of the anomalous information channel. "
As PEAR clearly state here, we have the FIDO data treated in a number of different ways, including binary, and the p-values are all similar. This shows that the drop in results is not due to the analysis method. Its odd that you post a quote that directly contradicts your hypothesis that the drop in results is due to the analysis method.
"Once again, there was reasonably good agreement among the six scoring recipes, but the overall results were now completely indistinguishable from chance. "
The results of the distributive trials were at chance level. This is a subset of results, not the overall results. PEAR reported overall significant results for their entire database (excluding the human judged Chicago trials). Also, these trials that used distributive descriptor questions, when analysed as binary answers, did not get positive results.
To summarise,
PEAR reported an overall highly significant result, contrary to what you claim.
Your hypothesis that the drop in results are due to the analysis method is not supported by the PEAR data.

