No, they just observe two main variables, one beeing an objective measure, amount of airflow, which is (very) highly significant - unusual in such small samples - the other one a subjective, which was not significant.
They obviously were successful in their obfuscation.
They were attempting to replicate a particular finding (as revealed by their power analysis) and failed to find a difference that was significant. They then present one of their other measures, in this case change in nasal inspiratory peak flow, in order to salvage their desire to conclude that homeopathy makes a difference. They had a choice of other measures - use of conventional drugs, ratings on each of a number of symptoms (interference with sleep, nasal symptoms, sneezing, eye symptoms, chest symptoms), adverse events - so it is not a surprise that they were able to find one that showed a statistically significant difference. The other studies included in the analysis show a similar pattern - multiple outcome measures with heterogeneity and inconsistency in the results. When you don't have a convergence of the data, this would normally be a cause for concern. Doesn't it make you wonder why there were no changes in perception of symptoms when there was a large difference in an 'objective' measure? Especially since other studies showed the opposite pattern or no relationship.
When you divide people into groups, you can always find differences between the two groups. Statistical analysis is used to discover whether the differences are unexpectedly large, but you have to pay close attention to avoid violating your assumptions of what should be expected (such as multiple comparisons). Study design is used to ensure that you can attribute these differences to a particular cause, but the presence of reasonable alternate explanations weakens your ability to conclude that a particular treatment was the cause.
I can not see any flaw here, because indeed,one of the main feats of hayfever is that you dont get enough air because everything is swollen due to the allergic reaction, so this seems reasonable to me, as a lay person. There is also no problem with more than one variable that defines any given construct (in this case the implicit construct of 'getting better'). The problem is more that they dont define their model very well. Also, they dont tell how much better this betterment is and that they do not control for other factors or check for interactions between the two dependent variables that they are using. (apparently, they are only using simple t-tests. I would have done a anova)
Also, certain statistical methods are indeed setup to test for many factors, for example in multi-factorial analysis, but they didnt do that here.
You state that you cannot see any flaw and then go on to list several flaws.
More than one variable can define a given construct, but one indication that your variables are providing a valid measure of that construct is whether or not variables that should be measuring the same thing actually show changes that are similar in direction and magnitude. If some measures go up, some go down, and some don't change, you obviously are not measuring that construct in a valid fashion. Simply selecting out those measures that went the direction you wanted and ignoring those which didn't after the fact is most definitely not the way to fashion a valid measure of a construct.
So this study is rather crude but alas, its not a full article, but only a small editorial.
That characterization is incorrect. It was not an editorial. It was a full report on a research study.
Conclusion: The outcomes are supported by the evidence. Clearly, a main symptom of hayfever, reduced airflow due to allergic reaction was significantly reduced in the verum group.
Unfortunately, study design and the statistical analysis do not support that conclusion. The p values attributed to the expectations are wrong as they depend upon erroneous assumptions, and the study design does not allow you to conclude that the differences were due to treatment as alternate explanations, such as group heterogeneity, are also potential reasons for group differences.
Well in the text, they had one center where people did worse, but overall, there was a significant difference in airflow. I cannot see manipulation here.
There are many different ways that the data can be analyzed from this study, and the potential for dozens of comparisons. It is suspicious to me that out of all those possibilities, the ones that happened to support the authors' conclusions were chosen. Especially when they admit that had they chosen different end points, the homeopathy groups would have performed worse than the placebo groups. There are also some glaring omissions in the discussion, such as why they went through the trouble of performing a power analysis and then only recruited less than half the number of subjects necessary to demonstrate a difference, or why they chose nasal inflow as a measure a priori. They are obviously leaving information out and you have to wonder why.
Hm, yes but statistics are as good as it gets. Of course, its significance is always questionable, but thats true for all studys using these methods.
The point is that p-values are not a direct measure of the strength of your conclusion, yet homeopathic researchers pretend (or don't know any better) that they are.
Also, you take as a fact that the the bulk of homeopathic research is too poor to draw any conclusions - well, i dont know, but i will investigate it further, to see if this opinion is justified.
I don't recommend it. My head almost exploded.
Yes, thats what effectiveness studys are for, to see if a certain method/compound has a positive effect in a certain situation (in this case,hayfever.)
Me neither. But again, i happen to use homeapthy for those every-day ailements, and most of the time it works for me. Only placebo effect? Well then, but it seems to be a good one, with no side effects whats or ever. And if it doesnt work,ok, i just take antibiotics or other suitable pharmceuticals. easy!
Cheers,
Roland
If the outcome is no different regardless of whether or not you take homeopathy, why bother? You are welcome to waste your own time and money. My complaint is with the waste of shared resources that could be better spent elsewhere.
Linda