Homeopathy: Rustum Roy drops the bomb

If it helps at all, here's a typical UV-Vis spec of four different vodkas.

[URL]http://www.internationalskeptics.com/forums/imagehosting/thum_838746cc359275365.jpg[/URL]

I can't imagine that either Rao or Hahnemann Labs would have used vodka.




Except when they were writing the paper....


OK, fair enough! It was just a thought, based on the fact that I know one homoeopath who does use vodka to do his dilutions. Of course vodkas tend to have additives, and quite a lot of them. Though I wonder about some sort of raw hooch?

Anyway, I checked all the ethanols in the Sigma catalogue, and only the spectroscopic grade (493511 for the 95% and 459828 for the 100%, also noted as HPLC grade), and the “USP/NF” (493538 for the 95% and 493546 for the 100%) have UV absorption data recorded.


Both strengths of spectroscopic grade have the same UV data, as we discussed earlier –
λ: 210 nm Amax: 0.40​

λ: 220 nm Amax: 0.25​

λ: 230 nm Amax: 0.15​

λ: 240 nm Amax: 0.05​

λ: 270-400 nm Amax: 0.01​

Both strengths of USP/NF also have the same UV data, as discussed earlier –
λ: 240 nm Amax: 0.40​

λ: 250 nm Amax: 0.30​

λ: 260 nm Amax: 0.30​

λ: 270 nm Amax: 0.10​

λ: 340 nm Amax: 0.10​
This latter looks like a relatively rough guide compared to the spectroscopic grade data, and makes me even more inclined to believe that it is something of this grade we’re seeing in the Hahnemann Laboratories samples.

As far as I can make out, USP/NF stands for “United States Pharmacopoeia - National Formulary”. Sounds exactly like what they might procure for their processes, and it seems to me quite possible that this might have a bit of between-batch variation in it. If they simply took the remedies off the shelf, and they were prepared at different times, then batch variation in the pharmaceutical-grade alcohol could easily explain what we’re seeing in these traces I think.

Er - just as JJM said about two days ago....

Rolfe.
 
Last edited:
Here is the first draft of the proposed letter. Constructive criticism please.

Dear Sir,

We wish to draw to your attention serious anomalies and incongruities in the UV absorption data presented in the paper by Rao et al., published in your August 2007 issue [1].

In a study of this nature, which in effect is examining multiple samples of ethanol, the over-riding concern must be absolute uniformity in the source of the solvent. For the data to be valid, it is essential that every drop of ethanol used must be sourced from the same stock bottle. However, the authors fail to make any mention of this point, and it is clear from the results presented that the source of ethanol was most certainly not uniform.

The most striking anomaly is the UV spectrum presented for “plain ethanol”, a single trace repeated three times in figure 3. The provenance of this sample is not recorded in the paper. This trace reveals extremely high absorbance (greater than 0.8 absorbance units) at 250nm, falling off steeply towards 400nm but still above 0.4 units by 350nm, and demonstrating an absorbance peak of 0.65 units with a l max of about 330nm. It is simply impossible to represent this trace as being ethanol of any recognised degree of purity. Spectroscopic grade ethanol has an absorbance of less than 0.05 units between 250 and 400nm [2], and even USP/NF pharmaceutical grade ethanol has an absorbance of less than 0.3 units at 250nm, falling off to less than 0.1 units by 270nm [3]. If the substance measured by the authors as “plain ethanol” was indeed ethanol at all, it is clear that it contained extremely high levels of impurities, possibly including acetone.

In contrast, the spectra of the samples which were diluted and succussed (Nat mur, Nux vomica and the “succussed ethanol” with no mother tincture), and which were presumably all supplied by Hahnemann Laboratories as detailed on page 178, demonstrate substantially lower levels of impurities. While still not being spectroscopic grade ethanol, these samples could well represent ordinary pharmaceutical grade ethanol. It is these samples the authors claim are “different”. However the evidence presented for this is weak to nonexistent.

Figure 1 presents one trace each for Nat mur and Nux vomica, each at 6C, 12C and 30C potencies. The traces are said to be “representative”, however with no information on repeatability or how the “representative” traces were selected, it is impossible to say whether there is in fact any real difference between any of the six spectra.

Figure 2 purports to address this point, but then fails to present the necessary data. The legend declares that 10 samples of each of the six remedy preparations were analysed. The normal way to present such data would be as mean absorbance ± standard deviation for each wavelength point, or at least for a representative selection of wavelength points. Statistical analysis could then be used to demonstrate whether or not there was a real difference between any of the remedies or potencies. However, the authors have chosen instead to present something they term “envelope of differences”. This is not a recognised method of handling such data, and it is not explained. We surmise that “extreme” high and low traces for each preparation were chosen to provide an impression of the range of results obtained. However, the fact that on two of the graphs the traces appear to cross may argue against this interpretation.

A further difficulty with figure 2 is that the upper (open circles) trace in the top graph of fig 2a (30C Nat mur) appears to be a duplicate of the upper (closed circles) trace in the top graph of fig 2b (30C Nux vom). Comparison with other traces of the two remedies indicates that this trace is really one of Nux vom, which has been duplicated into the Nat mur graph in error.

Paucity of data, ambiguity of presentation and lack of statistical analysis prevent any conclusions being drawn from the information in figure 2.

Comparison of figure 2 with figure 1 reveals that all six traces presented in figure 1 are taken from figure 2, in each case the closed-circles traces. If indeed the traces in figure 2 represent the extreme range of results obtained, this is startling, as the traces in figure 1 are stated to be “representative”. In addition, while it does appear that the Nux vom samples tended to demonstrate higher absorbances than the Nat mur samples (excluding the obvious mistake noted above), in two out of the three potencies the higher Nux vom trace from fig 2 has been chosen for inclusion in fig 1, thus exaggerating the apparent difference.

Figure 3 (b and c) again repeats the same six traces as figure 1, this time grouped by remedy. Presented in this way, it is clear that there is no difference between the three potencies of Nat mur, and that while variation between the Nux vom potencies is a little more pronounced, again all three traces appear to come from the same population. The same is true of the three potencies of “succussed ethanol” presented in fig 3a.

On simple visual inspection it does appear that there may be genuine differences between the three remedies (although no statistics are presented to allow this to be tested), with the Nat mur showing the lowest absorbtion and the Nux vom the highest, with the succussed ethanol lying somewhere between. Nevertheless, these differences are entirely consistent with small differences in purity of the ethanol stock used for preparation of the three remedies – small, that is, relative to the very high level of impurity evident in the “plain ethanol” sample presented alongside. This degree of variation in UV absorbance is entirely to be expected between different batches of pharmaceutical grade ethanol, which is not prepared with spectroscopic analysis in mind. The authors make no mention of having stipulated to Hahnemann Laboratories that all material sent to them should be prepared from the same stock bottle, and the data presented indicate that the different remedies, possibly prepared at different times, simply came from different bottles of ethanol.

We hope you agree that these are very serious points, and it is regrettable they were not identified by your own scrutineering process. It is clear that the data presented are wholly inadequate to support the authors’ assertion that UV spectroscopy can differentiate between the two remedies, and between different potencies of the remedies. If the authors wish to test their assertion so that it can be substantiated it will be necessary to repeat the work from the beginning, ensuring that all samples used in the study are sourced from the same bottle of stock solvent, that all duplicate preparations for precision assessment are separately prepared de novo from the mother tinctures, and that sufficient data are generated to allow robust and valid statistical analysis of the results.

Yours faithfully,

Rolfe
JJM
Wilsontown
Pipirr

References:
  1. Rao, M. L., Roy, R., Bell, I. R & Hoover, R. (2007) The defining role of structure (including epitaxy) in the plausibility of homeopathy. Homeopathy 96, 175-182.
  2. Sigma Aldrich catalogue, ACS spectrophotometric grade ethanol 95.0%, at www.sigmaaldrich.com/catalog/search/ProductDetail/SIAL/493511
  3. Sigma Aldrich catalogue, USP/NF grade ethanol 190 proof, at www.sigmaaldrich.com/catalog/search/ProductDetail/ALDRICH/493538


Rolfe.
 
Last edited:
I'd say it looks good. The only incredibly minor point is that the start of the fourth paragraph should read "In contrast, the spectra...".
 
Rolfe,

The letter is great, don't change anything. And thanks for writing it.
 
Well, thanks chaps. Calling Pipirr, now....

One extra little thought. I took literally the comment on Bad Science that a Google search on "envelope of differences" returned only references to people talking about Rustum Roy, and nobody knew what it meant. However, I thought I'd better check.

"Envelope of differences" gets very few returns, but three of them are not the Roy paper, and in these the term seems to mean something.

"Envelope of difference" gets a few more like that. Audio stuff, meteorology, crystallography and one relating to FTIR spectra.

The term does seem to have an actual meaning, and to be a legitimate way of handling some data. The trouble is, none of the links attempts to define it, it's just used in passing.

Whatever it is, it's still a totally illegitimate way of handling the data Roy says he has (the 10 repetitions for each remedy preparation). Nevertheless, I don't want to appear completely ignorant of something that has a legitimate definition.

Anybody care to see if they can do better than me searching the Google returns for a clue about what the term actually means?

Rolfe.
 
Can't contribute much to this discussion (though I have enjoyed reading it) - but I found a paper which defined envelope of differences for the purposes of their usage and I wondered if you could work out a definition from that?

http://www.riskmania.com/pdsdata/AMulti-factorStatisticalModelforInterestRates-ch4.pdf

Envelope of differences: the difference
between the position’s median Mark-to-
Future value and its potential future value at
the 97.5th or 2.5th percentile at each time step.​
 
I think this thread should be renamed;

Homeopathy: Rustum Roy drops the ball.

:D
 
Can't contribute much to this discussion (though I have enjoyed reading it) - but I found a paper which defined envelope of differences for the purposes of their usage and I wondered if you could work out a definition from that?

http://www.riskmania.com/pdsdata/AMulti-factorStatisticalModelforInterestRates-ch4.pdf


Bloody hell, Prof! That one redefines "incomprehensible"!

I still think all Roy means by the term is that he's picked the most extreme traces. Otherwise, why would the same traces appear in other graphs where there has been no attempt at duplication?

Maybe all I need to do is to make it less obvious that I don't know what the term actually means, and just comment that it's not defined, and not an appropriate way to present the data in question.

Such as -

However, the authors have chosen instead to present only two traces for each preparation, as "envelopes of differences", which is not an appropriate method of handling such data as most of the information is lost and statistical comparisons are prevented. The way in which these traces are derived is not explained.


And so on. This is a bit clumsy and maybe I'll think of a more elegant form of words in the morning. Mean time, all contributions welcome.

Envelopes of differences? Sheeeeshhhhh....

Rolfe.
 
I went through it with the paper in hand and have almost nothing to add. You were very thorough!

The only thing I would consider changing is:

Change this:
In a study of this nature, which in effect is examining multiple samples of ethanol, the over-riding concern must be absolute uniformity in the source of the solvent.


To this:
In a study of this nature, in which multiple ethanol-based samples are examined, the over-riding concern must be the absolute uniformity in the source of the solvent.

Or something like it. The original wording might be taken as a dig at the principles of homeopathy (it's all just ethanol!), by the sensitively minded...
 
I do not have the paper, but did not some of you mention that Roy discusses water properties at length, but he only analyses ethanol. Perhaps this criticism could be slipped in somewhere without over-playing our hand (and make the editor toss the letter in the bin)
 
I went through it with the paper in hand and have almost nothing to add. You were very thorough!

The only thing I would consider changing is:

Change this:
In a study of this nature, which in effect is examining multiple samples of ethanol, the over-riding concern must be absolute uniformity in the source of the solvent.


To this:
In a study of this nature, in which multiple ethanol-based samples are examined, the over-riding concern must be the absolute uniformity in the source of the solvent.

Or something like it. The original wording might be taken as a dig at the principles of homeopathy (it's all just ethanol!), by the sensitively minded...


I did think about that, but actually Roy never disputes that he's just analysing multiple samples of ethanol. That's his entire point. He thinks he's found changes in the structure of ethanol. Even though there could theoretically be traces of mother tincture in the 6C preparations, he's not claiming that physical presence of solute is having any influence. I can change it if you like, but I think it's probably not a contentious point in this context.

As far as other problems with the paper are concerned, how long have we got? There are a myriad other howlers and boo-boos and nonsenses in this. Wilsontown mentioned a reference that wasn't in the reference list (and what was there in its place didn't support the assertion being made). The entire structure of the thing is appalling - have the authors never heard of Introduction/M&M/Results/Discussion? At one point they claim they're going to present FTIR data, then don't - UV data are presented instead. The description of the materials and methods is a joke, with ludicrous detail about things that don't matter, and zippo about things that very much do matter. Much boasting about multiple runs and reproducibility, but no data on this - just the same few traces presented multiple times on different axes. And so on. (And I've a feeling that if we had anyone here who was familiar with Raman spectroscopy, we'd find that these results are just as laughably wrong as the UV ones.)

If we start to get into all this, it begins to look like a blunderbuss attack, and the letter loses focus. In fact, the other points are so obvious to anyone who knows anything about writing a scientific paper that I woudn't be surprised if Peter Fisher gets other letters just about these. And if anyone here wants to do that, don't let me stop them! But I think what we've got is a detailed dissection of the UV data, and we should focus entirely on that, even if it does mean we feel we've missed a few easy tricks.

Rolfe.
 
But I think what we've got is a detailed dissection of the UV data, and we should focus entirely on that, even if it does mean we feel we've missed a few easy tricks.

Yes, I think I agree with this. Pointing out all the problems with the paper would take another paper, and who has got time to do that? It seems reasonable to focus on the UV-vis stuff, because we're just about certain that it doesn't support the conclusions of the paper. This is the critical thing-that the conclusions (such as they are) are not supported by the data presented-and everything else is relatively minor.
 
I did think about that, but actually Roy never disputes that he's just analysing multiple samples of ethanol. That's his entire point. He thinks he's found changes in the structure of ethanol. Even though there could theoretically be traces of mother tincture in the 6C preparations, he's not claiming that physical presence of solute is having any influence. I can change it if you like, but I think it's probably not a contentious point in this context.

No, fair enough. You're right, the paper is all about the 'bulk properties' of ethanol itself, so it shouldn't be a problem.
 
OK, new paragraph 6, trying to address the "envelopes of differences" thing. ("Address the envelopes" - oh I'm such a wit....) I've taken out the sentence about some of the lines crossing, as it didn't really add anything.

Figure 2 purports to address this point, but then fails to present the necessary data. The legend declares that 10 samples of each of the six remedy preparations were analysed. The normal way to present such data would be as mean absorbance ± standard deviation for each wavelength point, or at least for a representative selection of wavelength points. Statistical analysis could then be used to demonstrate whether or not there was a real difference between any of the remedies or potencies. However, the authors have instead chosen to present only two traces for each preparation, as “envelopes of differences”. The derivation of these traces is not explained, although we surmise that “extreme” high and low traces for each preparation were chosen to provide an impression of the range of results obtained. This is not an appropriate method of handling data of this nature, as most of the information is lost and statistical analysis is rendered impossible.


Rolfe.
 
'Address the envelopes', cracking stuff.

I think the new version of paragraph 6 is fine. After all, it's really up to the authors to explain what on earth they mean, rather than for us to guess.

Just to draw your attention to this comment on the Bad Science journal club of the article, where someone who knows something about Raman spectroscopy weighs in.

http://www.badscience.net//?p=496 [look for comment No. 11]
 
'Address the envelopes', cracking stuff.

I think the new version of paragraph 6 is fine. After all, it's really up to the authors to explain what on earth they mean, rather than for us to guess.

Just to draw your attention to this comment on the Bad Science journal club of the article, where someone who knows something about Raman spectroscopy weighs in.

http://www.badscience.net//?p=496 [look for comment No. 11]


I thought as much. I did look at the question of different scales (and even different units) on the different Raman spectra graphs as shown in the PowerPoint, but I couldn't find anyone to explain it in detail - although someone did give a helpful guide to how it works.

From Another John, yesterday evening on Bad Science (Wilsontown's link)

Figure 4:
The range 750-975 cm-1 is only covered in Figure 4A as is the range 1560-1680 cm-1 making a mockery of their statement; “From the spectra shown in Figure 4, a clear distinction in the Raman active modes is noted between the TWO DIFFERENT REMEDIES as well as among the different potencies of the same remedy.”

Fig4A point (a) 6C and Fig4B point (a) 6C look to me to be the same intensity (1200 to 1300 arb units).

Fig4B point (b) 6C is probably no more of a peak then Fig4A point (d) 6C.
Looking at Figure 4 as a whole (bar Fig4A point (a) and Fig4B point (a)); every time a peak is highlighted the intensity around the peak is also greater. This is caused by just having a larger/bigger/fatter sample.**

Figure 5:
Assuming the peak positions a-f in Figure 5 to be real events, then I would suggest that the following “peaks” are not limited to the 6C sample as is stated by the authors:
a) there is a shoulder at around 275 cm-1 in all the samples .
b) the shoulder at around 380 cm-1 is definitely present in the “plain” sample (I admit there might be a slight shift in wavelength) and possibly in the 12 and 30C samples as well.
c) the shoulder at 415 cm-1 is present in all the “plain” sample. It is a little less intense, but then so is the main peak (probably due to different quantities of sample being scanned).
d) In the 6C sample, this peak at 625-630 cm-1 is around the same intensity as the peak at 650 cm-1. This is also true for the “plain sample”
e) is present in all the samples.
f) there is a shoulder in the plain sample at the same point as the peak.


I couldn't cope with adding a critique of the Raman spectra to what we've got, partly because it would be too long, and partly because I'd just be parroting - I know nothing about Raman spectra.

However, I wish Another John would come by here and elaborate. Maybe a separate letter with different signatories could be prepared, dealing with the rest of this duff data.

I think we need to give special, extra thanks to JJM though. It was he who spotted the filthy dirty nature of the "plain ethanol", and pointed us all at the Sigma catalogue specs for what ethanol should look like at 250-400nm. Nobody else seems to have picked up on this at all, and it's the absolute killer argument.

Rolfe.
 
I mentioned how few (non-Roy) returns I'd got when Googling "envelope of differences". Just enough to suggest Roy hadn't made it up, but very few indeed, and mostly very vague, almost as if the term just meant whatever the writer wanted it to mean at the time.

I just Googled "standard deviation". 24,300,000 returns, with the entire first page being links to pages defining the term.

Roy doesn't seem to have heard of it though....

Rolfe.
 

ISF - Join now!

Every member here is approved by hand. No bots, no spam, just people who care about evidence and honest debate.

Membership is free!

Create your free account

Back
Top Bottom