Designing a test for the magnetic coasters...

cogreslab said:
Hock: I have now responded to the Harmoniser question on the bioelectromagnetics page,
Click to expand...
Actually, no you haven't - you've merely denied knowledge of your own "experiments".
q.v. Thanks Paj, I look forward to seeing the protocol and results in due course.

Btw I abbreviate others' pseudonyms for parsimony only, and not in any way as an intended pejorative. Hope you can live with these shortforms.
Click to expand...
No, Cogs, I can't. YOU can refer to the bioelectromagnetics pages for that response.
 
I intend to use the protocol designed by the members on this thread. Blinded, randomised plastic cups. Groups of 2 normal and 1 treated water. Participants won't know what it is they are testing. They will seek to identify the "different" water, regardless of whether they think it is better or worse. They will not have contact with anyone who knows which cup is which.
 
How many people and what age groups/sex/socioecon status?in your sample Paj? You mentioned school. Are they school children?

Don't get distracted by Hock's typically rude invasion of this thread!
 
I'm planning to use the staff, actually Roger.

Why is it a concern? Do your coasters have better success with people from a certain demographic? :confused:
 
cogreslab said:
How many people and what age groups/sex/socioecon status?in your sample Paj? You mentioned school. Are they school children?

Don't get distracted by Hock's typically rude invasion of this thread!
Click to expand...
The name is Hocking, Mr.Coghill. Who's being rude here? Oh, and as for "invasion" of the thread.
1. This is a public thread - anyone can participate
2. I have been participating on this thread much earlier than yourself and have been inputting to discussions of test protocols as well as volunteering to participate in the actual test.
3. Quit with the ad homs. If you have objections to repeated demolitions of your "experiments" (your dog bowl "experiment" on this thread, from memory), why don't you address the criticisms.

Play the ball, Mr.Coghill, not the man.
 
A quick question, for anyone who has a view.

In the detailed protocol outlined on this thread, a method of randomisation was used which balanced the position of the magnetised water across all the trials. Participants were encouraged to taste the drinks in strict order, to remove the effects of position. But is this strictly necessary?

Surely, if a participant suspects that the magnetised drink will be evenly spread across their own trials, they may alter the results to suit. People tend to do this all the time anyway, when facing multiple choice options. "Well, the last one was B, so it can't be that again"

So, what's wrong with having TRULY randomised positions? Especially if the participants were encouraged to go back and taste each of the three samples several times before making a decision. Wouldn't that eliminate a possible "first cup tastes always freshest" bias?

Also, because of the nature of the environment when I plan to conduct the trial, I may get extra participants joining in half way. I would like to be able to add participants and automatically randomise their positions on the fly.


Any thoughts on this?
 
TheBoyPaj said:


So, what's wrong with having TRULY randomised positions? Especially if the participants were encouraged to go back and taste each of the three samples several times before making a decision. Wouldn't that eliminate a possible "first cup tastes always freshest" bias?
Click to expand...

In a word, no. The reason is, it's possible for the random numbers to conspire against you so that the first cup always happens to be the magnetized water. If you really think that the first cup will always taste the freshest, then you need to control for it by making sure that there isn't a correlation between the first cup and the magnetized water.

Of course, it's unlikely that the random numbers will conspire against you. But even unlikely things happen from time to time (by definition) unless you explicitly exclude them. And if you don't expect it to be a big issue, you can cross your fingers and ignore the effect.
 
No special reason, Paj, for any concerns about the group demographics. I wondered whether an adult sample might take the test more seriously, that was all. I think you might need to do a few calculations to ensure that the sample is big enough to detect a 15 to 20 percent difference. It might be wotrh asking a professional statisitician for his views on the study design too. Any nearby University should have someone with that expertise.

Maybe I should give you more than one coaster? If that helps, just let me know, (since the other group I supplied don't seem to be very active just now), and I will supply you with some more.

We are going to run our own trial with a dozen coasters and a few hundred subjects, but we're so busy right now that I can't see it happening this year.

One other thought: plastic cups might impart a taste to the water while glass would not. Have you thought of that?

Best, Roger
 
New Dr Kitten:

Thanks. In that case I think I will prepare a randomised set where the frequency of each position is equal overall , but they are not quite as evenly distributed as in Steve's proposed scheme, to prevent a participant guessing the pattern. An individual may still get lots of A's, for example. But across the set it should even out.


Roger:

I'm planning to use paper cups, for that reason.
 
cogreslab said:
No special reason, Paj, for any concerns about the group demographics. I wondered whether an adult sample might take the test more seriously, that was all. I think you might need to do a few calculations to ensure that the sample is big enough to detect a 15 to 20 percent difference. It might be wotrh asking a professional statisitician for his views on the study design too. Any nearby University should have someone with that expertise.

Maybe I should give you more than one coaster? If that helps, just let me know, (since the other group I supplied don't seem to be very active just now), and I will supply you with some more.

We are going to run our own trial with a dozen coasters and a few hundred subjects, but we're so busy right now that I can't see it happening this year.

One other thought: plastic cups might impart a taste to the water while glass would not. Have you thought of that?

Best, Roger
Click to expand...
Care to share your methodology for this experiment with us. If you scan this thread you will see a number of interested parties that could tighten up any procedures or advise on statistical analyses etc.

As you say - we have months to iron out a protocol before you perform this experiment on over 300 people. BTW - how do you intend to manage to test such a vast number?
 
cogreslab said:
No special reason, Paj, for any concerns about the group demographics. I wondered whether an adult sample might take the test more seriously, that was all. I think you might need to do a few calculations to ensure that the sample is big enough to detect a 15 to 20 percent difference. It might be wotrh asking a professional statisitician for his views on the study design too. Any nearby University should have someone with that expertise.
Click to expand...

Um, a 15-20 percent difference in what? Meaning you expect
48% of the people (33%+15%) to pick the magnetized water, and 26% (each) to pick one of the other two in the triangle format? The way the triangle is test is set up, it's not going to be able to measure stuff like "the water tastes 15% better," even if it were true.

Of course, Paj, Roger is correct that you might want to reality check to make sure that the power of your test is sufficient to pick up the expected effect size. Given how many subjects you plan to run, how many correct identifications would be necessary for a "significant" result?

Also : some of your issues with randomization might be addressable by a drawing-without-replacement scheme. Make a deck of thirty or so cards, with equal numbers of "mag. water first," "mag. water middle," and "mag. water last." Each time you run a single trial, draw a card to determine the order to present, but don't reshuffle until you get to the end of the deck. This will limit how out of line the distribution can possibly get,... and the only way for someone to guess the pattern is by card-counting. If they're good enough to do this, to heck with the woo-woo water -- take them to the casino and get some real money.
 
That's a good idea. I'll give it a shot, though probably substituting a computerised "deck" for a physical one. I don't fancy the paper-cuts from all the shuffling that would be necessary!

Regarding the trial size, I anticipate getting at least ten participants, taking 6 tests each. I may have to split this over a couple of lunchtimes due to different people's work commitments, but I can't imagine this would affect the outcome so long as the protocols were the same on each sitting.

After all, the instructions don't say anything about it working only on certain days!
 
Paj, Im not sure ten subjects would give you enough statistical power. If I send you another couple of coasters would that help you increase the number of testers?
 
new drkitten said:
In a word, no. The reason is, it's possible for the random numbers to conspire against you so that the first cup always happens to be the magnetized water.
Click to expand...


Yes, it is possible. But if, when you go through a randomization procedure and dislike the random arrangements that result from it, you really need to question your randomization procedure.
 
cogreslab said:
Paj, Im not sure ten subjects would give you enough statistical power. If I send you another couple of coasters would that help you increase the number of testers?
Click to expand...
Why are you not sure ten subjects would be sufficient?

How many do you suggest.

And why?

What was the deciding factor in selecting 300+ test subjects?

And what is the methodology you intend to apply to this test?

C'mon, you're the scientist with the £1MM lab. You should be advising us on best protocols to test your device.
 
jzs said:


Yes, it is possible. But if, when you go through a randomization procedure and dislike the random arrangements that result from it, you really need to question your randomization procedure. [/B]
Click to expand...

Of course. But I prefer to question the randomization procedure before running the test, as it saves me the trouble of running the experiment twice.
 
new drkitten said:
Of course. But I prefer to question the randomization procedure before running the test, as it saves me the trouble of running the experiment twice.
Click to expand...

Yup. I agree.

I think that it is doubtful that the randomization would make it so "that the first cup always happens to be the magnetized water" however.
(underline mine)

As you say, it is unlikely.
 
Update:

I'm having trouble finding a source of cheap paper cups!

Plastic would be no problem. Foam is a possibility. But paper? No way.

I could get a packet of about 5 party cups (with Shrek or Spiderman or some such character) for about a quid, but that would make for a rather expensive experiment.

So, I apologise for the delay and I'll continue looking.
 
You must log in or register to reply here.

Back
Top Bottom