T'ai Chi said:
You and Claus are asking me to explain how someone uses their brain to select something? Um..
Look, if an evaluator knows deep in his mind that one method generally gives smaller p-values than another method, he might unconsciously tend towards that one (or another one) if he desires to unconsciously support his bias as much as he can.
No no no. Let's recap a bit here, since you're clearly getting lost in your own poppycock.
o You keep doing this side-slip of smearing the roles of evaluator and statistician.
o You said the test selection bias can occur before the data have been analyzed. Hence, the paranormal questions which you need to address, sir.
o If you wish to withdraw that paranormal claim, you are now stuck claiming the statistician looks at the data twice. The first time, he doesn't get the result he wants. He then would change tests at least once to get the result he wants. This is no longer bias, but outright scientific fraud.
o "Generally gives smaller p-values" is only true for certain test types over certain data ranges. If you maintain this an
a priori situation, you again are making paranormal claims. If
a posteriori you again have scientific fraud. More probably, the statistician has selected an inappropriate test, which would be caught in review torn apart by other researchers following publication.
o How much gandy-dancing must you do with your alpha levels to make all this magic happen? Why not just make your bullsh!t transparent and set alpha to .026 to make sure the test falls on the right side of the line? Why not add a few data so long as you're already being "unconsciously" fraudulent? Just to make sure p falls on the correct side of the line?