Jack by the hedge
Safely Ignored
- Joined
- Oct 14, 2009
- Messages
- 23,415
I wonder if a different approach might be valuable.
It is an interesting question as to how to design a valid statistical experiment in which there is an issue about the credibility of the answers. (There has been at least a little discussion about this in the past.) As someone who doesn't do experimental design, I would be interested in learning how this might be done. If a design appeals to Michel, fine. If not, also fine.
As an initial suggestion, I think one could design a setup where both responses and any accompanying comments go to a third party. The third party then tosses a coin and either changes the specific response or leaves it as submitted, keeping track of which were changed. Everything then goes to the person who evaluates each response and who can decide which ones are credible. Those considered credible are then revealed to all. It ought to be possible to back out a valid statistic I would think.
That's an interesting idea. To get around having to trust the third party, they could share a version of the original answers (plus a few random letters to stop its being guessable) scrambled with an MD5 Hash generator, and only reveal the key to decode it later.