Designing a test for the magnetic coasters...

[Pragmatist]

What's to stop them now advertising them with the phrase: "As tested by the JREF forum"?

Well, once the tests are done. You know what I mean.

The point is that anyone can make a claim sound good to those who cannot be bothered to question the implication.

A simple contract. Drawn up and signed and agreed by both parties BEFORE the test.

Suggested wording: "Both parties agree that the results of this test shall not be used for any form of sales advertising unless the wording of any such advertisement is approved in writing by both parties prior to said advertisement being made"

A lawyer could do better, but you get the point.

And I agree with your second point. But all of us, for one reason or another tend to occasionally accept something improbable at face value, it doesn't make anybody particularly lazy or stupid, it's just that we're all human and fallible. I tend to feel that those who are NOT lazy or stupid have at least some responsibility to protect their err..."less energetic" brethren shall we say!

 

[Pragmatist]

However, even if we test (and report) that the water tastes revolting, the exact same
advertising blurb : "even hardened skeptics associated with the James Randi Educational Foundation were forced to admit that they could taste the difference!" is still TRUE and could still be used. We admit that we could taste the difference. Even if we tested the water to look for magnetism, they can still advertise "Our claims have been tested by hardened skeptics!," without mentioning that no results were found in our tests.

But a pre contract could avoid all of this by simply stating that no reference to the test is to be used in advertising without prior agreement of the contents by all concerned. It doesn't have to be anything complex or fancy.

Note the scare quotes around the word 'magnetize.' In the same way that I can autoclave a scalpel (expose it to an autoclave), I can also magnetize something by exposing it to a strong magnetic field. This is, in fact, one of the standard uses when one talks about electromagnets; one 'magnetizes' them by turning them on, and when the current is turned off, they no longer display magnetic properties.

Mr. Coghill's claim is not that he can cause water to become magnetic. It is that by exposing wine to a magnet, it will taste better.

We'll have to agree to disagree on this one.

On the bioelectromagnetics thread, when specifically asked a direct question about whether this "magnetising water" was unique to HIS magnet or could be performed by ANY magnet, he claimed that it WAS unique to his magnet because he had some scientifically objective way of measuring the alleged effect and thereby "tuning" the magnet for optimum effect.

The quotes around the word "magnetize" clearly indicates that the word is not to be taken literally, that one cannot simply assume that it is ordinary magnetization. And of course as I have previously explained, water is diamagnetic and cannot be "magnetized" in the ordinary sense of the word anyway.

Therefore, in response to a simple enquiry he made a related claim. And I don't see why OUR enquiries are any less valid than those of an ordinary buyer.

I maintain that it is up to Roger to clearly define what he means by the word "magnetize" in this context and to reveal the objective test that he insists will prove the claim he makes.

 

[drkitten]

I maintain that it is up to Roger to clearly define what he means by the word "magnetize" in this context and to reveal the objective test that he insists will prove the claim he makes.

You are assuming a degree of cooperation that may not be forthcoming. If he does not provide such a definition and such tests, what then?

 

[Pragmatist]

You are assuming a degree of cooperation that may not be forthcoming. If he does not provide such a definition and such tests, what then?

Well then the only possible conclusion is that the claim is not proved because the claimant has failed to provide sufficient information to enable it to be tested.

It comes back the earlier question I mentioned, why are we doing this? The answer as far as I am aware is that Roger offered these things and said, "if you don't believe my claims, test them yourselves and you'll see that they're true".

If he then frustrates any reasonable attempt to actually DO a test, then what are we supposed to conclude? It's in his own interest therefore to cooperate, after all it was HIM who suggested the test in the first place.

 

[steve74]

It comes back the earlier question I mentioned, why are we doing this? The answer as far as I am aware is that Roger offered these things and said, "if you don't believe my claims, test them yourselves and you'll see that they're true".

In this case it was Tez who proposed the test of the coasters. The problem with Coghill providing his protocols and results is that if the tests were replicated and showed negative results (as I believe they almost certainly would) is that there would then be two conflicting sets of results.
Coghill is advertising these products as being fit for a particular purpose - namely that they will improve the taste of water and wine. If a protocol can be devised that tests this claim and is acceptable to Tez and Roger (who are putting their money on the line) then this is the claim that should be tested. Yes, the claim is subjective which means the testing will also be subjective to a certain degree, but drkitten's suggestion of a forced choice triangular test minimises this problem.

I agree that both parties should have agreement on the protocol of the test, its scientific 'validity' and what claims can subsequently be made about the results.

The reason I think this test is worth doing is that Coghill is using pseudoscience (as you have so ably demonstrated in the Bioelectromagnetics thread, Pragmatist) to sell worthless junk to the public.

 

[Pragmatist]

In this case it was Tez who proposed the test of the coasters. The problem with Coghill providing his protocols and results is that if the tests were replicated and showed negative results (as I believe they almost certainly would) is that there would then be two conflicting sets of results.
Coghill is advertising these products as being fit for a particular purpose - namely that they will improve the taste of water and wine. If a protocol can be devised that tests this claim and is acceptable to Tez and Roger (who are putting their money on the line) then this is the claim that should be tested. Yes, the claim is subjective which means the testing will also be subjective to a certain degree, but drkitten's suggestion of a forced choice triangular test minimises this problem.

I agree that both parties should have agreement on the protocol of the test, its scientific 'validity' and what claims can subsequently be made about the results.

The reason I think this test is worth doing is that Coghill is using pseudoscience (as you have so ably demonstrated in the Bioelectromagnetics thread, Pragmatist) to sell worthless junk to the public.

Oops, yes, you're absolutely right. My apologies, faulty recollection. I also forgot that Tez had money on this.

My biggest worry is simply that this could backfire if someone tricky wanted to take advantage of it. If you've been following biolectromagnetics you'll probably understand why I think that!

 

[EHocking]

True enough, but I still don't like the idea of testing taste. It's just too subjective. The problem is that even if there IS some difference in taste, we have to trust totally that each person involved will report honestly what they perceive. We have no objective way of determining what somebody else "tastes".

Perhaps we could complement the subjective test with some degree of objective measurement?
There are a number of testing equipment and methods listed here;
<http://www.homebrewheaven.com/professional-wine-testing-kit.htm>
A fair number of them would appear to be fairly straightforward chemical tests (ie pH, specific gravity). Rather than purchasing any tools, such as the the Brix refractive device, might Tez have access to simple testing equipment such as that listed on this page?

For a wine to taste "better" or even different, you'd have to expect to see a chemical difference in the wine(?). Analysis of the results could merely be a determination of whether any chemical differences are beyond experimental error and are statistically significant.

 

[drkitten]

Perhaps we could complement the subjective test with some degree of objective measurement? There are a number of testing equipment and methods listed here;


For a wine to taste "better" or even different, you'd have to expect to see a chemical difference in the wine(?). Analysis of the results could merely be a determination of whether any chemical differences are beyond experimental error and are statistically significant.

Unfortunately, the number of ways in which two substances could differ exceeds the number of chemical tests it's practical to run. Basically, chemical assay is really only reliable if you know what you're looking for, and otherwise it's simply a hit-or-miss proposition.

That's the problem with these "objective" tests. They are quite likely to be measuring the wrong thing. That's why I keep repeating that we should focus on Mr. Coghill's claim that it improves the taste....

 

[drkitten]

Well then the only possible conclusion is that the claim is not proved because the claimant has failed to provide sufficient information to enable it to be tested.

Then, how did we manage to develop a protocol that tests the claim?

There's an even easier testing protocol if we had enough subjects : give five hundred people blinded samples of both and ask which tastes "better," then score. But we don't have those resources to hand, so the triangle test is more appropriate in this instance.

 

[EHocking]

Unfortunately, the number of ways in which two substances could differ exceeds the number of chemical tests it's practical to run. Basically, chemical assay is really only reliable if you know what you're looking for, and otherwise it's simply a hit-or-miss proposition.

That's the problem with these "objective" tests. They are quite likely to be measuring the wrong thing. That's why I keep repeating that we should focus on Mr. Coghill's claim that it improves the taste....

Point taken. Perhaps then, there is someone on the Forum that may know someone in a wine club or similar. With their cooperation the test could be run with wine "experts" as the subjects and they would not need to be informed as to *what* was being tested. You could just tell them that we wanted to conduct a taste test between two red wines.

 

[cogreslab]

Just my opinion, but I think this is a big mistake.




At the very least, I think it would be essential in this case, if the taste test is to proceed, to obtain from Roger a signed agreement, preferably legally notarized, to the effect that the proposed "test" is just for fun. The agreement should explicitly state that the test is in no way scientific, nor that it constitutes proof of anything. Roger should also agree that he will not use the result of this test in any way to promote any product or to make any claims based upon it.

My fear is that anything less just plays right into the hands of potential deceivers.

The first and foremost question we should all be asking is, what do we hope to achieve with this test, and why is it being performed?

I'm not trying to preach here, just trying to explain clearly why this proposed test worries me personally.

I am putting on record that I, Roger Coghill, will not exploit in any way for commercial gain or for any other purpose the findings of your testing of these magnetic coasters, whether postive or negative, and I will disclaim publicly any similar exploitation by the manufacturers/distributors of the said coasters.

I hope you will accept this recordable statement since going to the trouble of having it notarized is overkill, imho.

 

[drkitten]

Point taken. Perhaps then, there is someone on the Forum that may know someone in a wine club or similar. With their cooperation the test could be run with wine "experts" as the subjects and they would not need to be informed as to *what* was being tested. You could just tell them that we wanted to conduct a taste test between two red wines.

Using a hypothesis-blinded subject pool would be one way (probably the best way) of controlling for potential bias in the tasters. As such I think this would be a good idea. Of course, this also introduces issues of representativeness -- the tastes of the wine "experts" may not be representative of the tastes of the general public. But more generally, I'm not sure it's practical, because Tez, the man with the coasters, may not know such a club.

One of the unfortunate realities of experimental science is that you work with what you can get.

 

[cogreslab]

There's an even easier testing protocol if we had enough subjects : give five hundred people blinded samples of both and ask which tastes "better," then score. But we don't have those resources to hand, so the triangle test is more appropriate in this instance.

Do not forget my offer to supply more coasters if that helps, e.g. to test them in more than one geographical location.

 

[TheBoyPaj]

Would you be willing to supply some to me, so I can run the same test in the North of England?

 

[T'ai Chi]

In my opinion, the problem is even worse if any fancy statistics are going to be used.

A test of proportions and basic experimental design is "fancy"?

Because that will also add a veneer of "science" to the whole affair which is anything but scientific.

If you have another way of analyzing data besides using statistical theory and methods, please, let me know.

The first and foremost question we should all be asking is, what do we hope to achieve with this test, and why is it being performed?

We're seeing if there is a statistically significant departure from the expected p = 1/3 proportion. ie, to see if there is evidence for the claims for the coasters' efficacy.

 

[EHocking]

You're right! My cat only drinks running water. She jumps to the wash basin and meows until someone opens the fawcet for her.
Anyway, since pets can't give deposition, a test would only be valid if witnessed by a team composed of skeptics and believers.
Photos would not be proof, since it could be allegged that the bowls contained different liquids.

Well there is only so much you can do - obviously the subject animal would need to have fairly "normal" feeding habits, which would need to be observed prior to testing the "magnetised" water.

You would wish to do this as a control anyway by emulating the test conditions and to determine if there is a feeding preference by the animal. You can then compare the test results against that of the control.

It should not be too difficult to determine the "randomness" of the control and the threshold that the experimental results would need to achieve that of significance over the control preferences of the animal.

 

[EHocking]

Using a hypothesis-blinded subject pool would be one way (probably the best way) of controlling for potential bias in the tasters. As such I think this would be a good idea. Of course, this also introduces issues of representativeness -- the tastes of the wine "experts" may not be representative of the tastes of the general public.

Since suggesting it, I've been trying to figure how you might be able to approach a club. As it happens, I do know a couple that are in a wine appreciation club and are learning the finer points of tasting from this club.

Perhaps we could present the test through the club, as an exercise for honing the tasting skills of a group of novices in the club? The score sheet would have to be changed to add in, perhaps, a column for the subjects to add in tasting notes? Or even to reproduce a standard that the club uses, but with the triangular protocol worked into it.

I'd suggest a novice group primarily because they are more likely to be representative of a wine drinking general publice. The problem with using expert tasters, in my mind, is that if a less than premium wine is used it is more probable that an expert taster would pick that the wine samples are the same wine.

An alternative, less controlled experiment, or precursor experiment, might be to merely set up a table at one of their regular tastings and invite the group to test the three samples at their leisure. A screened off "servery" could be set up behind the table where the samples were prepared randomly for each "volunteer" as they approached. Later in the evening we could present to group the purpose of the sampling table and the preliminary results.

It's not as ideal as having the same volunteers score multiple trials, but might be useful for generating a larger population of scores.

But more generally, I'm not sure it's practical, because Tez, the man with the coasters, may not know such a club.

One of the unfortunate realities of experimental science is that you work with what you can get.

True, but there are a number of clubs in London, it may be possible through my contacts to find a club that is "initiating" some novices that is convenient to Tez's location and calendar.

Also, wine clubs are not ignorant of the claims of these devices. I found a couple of editorials by website correspondents that are informally testing a device called a "Wine Clip" (from memory). I'll try to find time to drop them an email to see how they've approached it.

By dint of this post, I guess I'm volunteering to help him out where possible!

 

[Tez]

Ok, I think we have a pretty workable protocol based around steve's post (thanks for putting the time in for that), which is of course a product of many of your much appreciated inputs.

I'm pretty much prepared to do it any weekday afternoon/evening this week or next. EHocking, I am glad you've agreed to volunteer!

FWIW I agree with Pragmatist in many of his suggestions, in as much as were we simply scientists setting out to explore the realm of the effects of magnetisation on various liquids, this is certainly not the sort of test we'd be doing! However, the nature of this test is partly due to the history of Roger's and my interactions...

(BTW, what "sort" of Pragmatist are you? When forced to make a statement on my philosophical tendencies I generally claim to ascribe to some version of "old" pragmatism. Personally I find Rorty goes too far for me. Davidson I like a lot, and indirectly he has influenced the way I think about quantum mechanics and realism in general. Rorty has as well to be truthful )

 

[cogreslab]

Would you be willing to supply some to me, so I can run the same test in the North of England?

Sorry to be slow in responding. I didnt see your question until now. Yes, of course. How many do you want me to send to you?

 

[EHocking]

Ok, I think we have a pretty workable protocol based around steve's post (thanks for putting the time in for that), which is of course a product of many of your much appreciated inputs.

I'm pretty much prepared to do it any weekday afternoon/evening this week or next. EHocking, I am glad you've agreed to volunteer! *snip*

Just need a few day's notice to organise myself - evenings only I'm afraid. I guess, also, I'd only be of use as a lab "tech", since my responses as a lab "rat" could possibly be coloured by my participation here.