Circumstitions

[fls]

It will take me a few days to reply.

I've got several hurricanes breathing down my neck. If you wait too long you may lose your chance.

Linda

 

[Shuggy]

The issue of presenting misleading and inaccurate information in order to support a particular viewpoint was raised on another thread about circumcision. I specifically referred to circumstitions.com as one of these sites, which prompted a response from someone who seems to have some involvement in the site.

I am starting a new thread with some examples, so as not to derail the original thread.

I am using this page as an example:

The following statements are misleading.

Research can have clinical significance (e.g. the outcome is important, the effect on that outcome is substantial) and/or statistical significance (it is very unlikely that we would see this difference if it was due to chance). Making a point of stating that the significant differences are statistical implies (otherwise why even bring it up?) that the differences are not clinically significant. Yet, by any reasonable measure (prevention of a transmissible disease with a 100% mortality, ~50% relative risk reduction, regions with double digit prevalence), the outcome is also clinically significant.

That seems a reasonable point and I will (re)move the reference. I was overgeneralising my reaction to the hype worldwide about circumcision causing "60% reduction in HIV" to the point where an Israeli paper (and I've been unable to find it again) interpreted that as "Six out of 10 circumcised men are are immune to AIDS". There have also been cases (www . circumstitions . com / news / News20.html #hiv-female") where non-significant results are quoted as "indicating a trend towards" some claim or another, and the lack of significance is omitted from the news stories.

Raising this point implies that the authors are attempting to mask the size of the results. Relative risk reduction and absolute risk reduction are two different ways of summarizing information in a useful manner. That the number for relative risk is always larger than the number for absolute risk simply reflects that they are different measures, not that using one number masks information from another. If anything, the use of absolute risk reduction masks the results, because it cannot be applied to any other group when the underlying incidence varies - exactly the situation we see in spades with HIV. The authors chose to use a summary measure that was useful instead of a summary measure that was not.

The fact that the absolute risk reduction is small also means that small inaccuracies have greater significance. "60% [now being hyped up to 70% or - out of nowhere - 80%] reduction" sounds pretty unarguable but if you know that in total only ~137 controls vs ~64 experimental got HIV out of 5,400 of each, it's not so impressive, when you consider they they ignored the possibility of non-sexual or same-sex transmission, the much greater number of dropouts, and so on.

No attempt is made to demonstrate that these criticisms apply or that they would negate or influence the outcome. The experiment was double-blinded to the full extent possible.

I raise it because circumcisionists are saying "RCTs are the gold standard" when the gold standard is placebo-controlled, double-blinded RCTs. (Nor were these truly random in the sense of a random population sample: all were paid volunteers who wanted to be circumcised.) Of course it's not possible to conceal from the subjects who is being circumcised, but it is possible to do a dummy operation that is similar, so many of the issues around one group having an operation and the other not are removed.

Measures that were performed unblinded were objective (HIV serology) so that unblinding would be unable to affect the outcome - making the criticism irrelevant.

This is naive. Experimenter and experimentee effects apply all over the place. In other studies (such as ESP work) it is well known that innocent (and not-so-innocent) mistakes tend to be made in the direction the experimenter wants the experiment to go, and these experimenters (and the subjects) make no secret of their desire for circumcision to be protective. The circumcised group was given additional safe-sex promotion that the control group was not.

Statistical reasons for an apparent decline in effectiveness or for the masking of a real decline in effectiveness were not applicable in this experiment. Appropriate use of statistical methods requires the selective use of statistics when considering validity and reliability.

I need more detail about that.

The following statements are inaccurate:

Since the study included the promotion of safer sexual practices, it demonstrated that substantial benefit could be seen in addition to the promotion of safer sexual practices.

As above, the promotion was not identical for the two groups (as it would have to have been if they could have been blinded). Circumcision can only be effective where safe sex is not practiced, ie where condom-promotion is ineffective. The more effectively condoms are promoted, the less effective circumcision must be. Wear a condom and the foreskin becomes moot.

You can go to this site and see that there are many accepted treatments that have a higher NNT. For example, the number of people with hypertension who need to take aspirin in order to prevent one heart attack is 176, yet this is an accepted treatment.

Well that surprises me, and it suggests that aspirin is being overhyped as protective against heart attack, and the only reason it is promoted is that it is so proverbially harmless. I went to that site, clicked on "infectious diseases" and for treatments of HIV/AIDS, the NNTs were all under 21 and most in single digits.

The more important consideration is the time period. You only need to circumcise men once in order to save one life per year for the next twenty years. If you want to see the same benefit in other accepted treatments, you have to treat those people twenty times. Once you inflate the NNT for accepted treatments by twenty (or reduce it for circumcision by that amount), the NNT for circumcision compares very favourably with other treatments.

I'm not sure if that is statisically accurate. It assumes that the protection is uniform over time, and if you look at the charts of actual cases offered in the original papers, the numbers are so small that it's impossible to say if it is.

Humans are not laboratory rats, and it may well be impossible to perform the kind of scientific experiment on them that one would like. One can't for example control when and where and with whom they have sex. It seems to me that these experiments were carried out with the same assumptions that can be used on lab animals, and the same confidence is now being placed on the results.

There is also the Hawthorne effect, where behaviour changes just by the fact of being involved in an experiment.

The other unfactored-in aspect is the extraordinary cultural role of circumcision, both for the subjects and for the experimenters. This, I suggest, greatly magnifies the importance of experimenter and experimentee effects.

 

[fls]

That seems a reasonable point and I will (re)move the reference.

Thank you.

The fact that the absolute risk reduction is small also means that small inaccuracies have greater significance. "60% [now being hyped up to 70% or - out of nowhere - 80%] reduction" sounds pretty unarguable but if you know that in total only ~137 controls vs ~64 experimental got HIV out of 5,400 of each, it's not so impressive, when you consider they they ignored the possibility of non-sexual or same-sex transmission, the much greater number of dropouts, and so on.

They did not ignore the possibility of non-sexual or same-sex transmission, or the number of dropouts and so on (I don't know why you said "much", since the number of dropouts in each group was essentially the same). All seroconversions for HIV were included, regardless of mode of transmission.

I agree that small numbers can lead to problems with generalizability. However, there are now 3 large RCT's and numerous epidemiological studies on this issue which show a consistent effect.

I raise it because circumcisionists are saying "RCTs are the gold standard" when the gold standard is placebo-controlled, double-blinded RCTs.

The problem with making absolutist statements is that there are inevitably situations where they don't apply (or at least don't apply well). Rather than making these kinds of statements, I think it is better to specify the relevant details, especially since this is one of those situations.

(Nor were these truly random in the sense of a random population sample: all were paid volunteers who wanted to be circumcised.)

"Randomized" in the context of RCT does not refer to sample selection, but to treatment assignment. The participants in this trial were appropriately randomized. Potential participants in a clinical trial are not chosen on the basis of a random population sample.

Of course it's not possible to conceal from the subjects who is being circumcised, but it is possible to do a dummy operation that is similar, so many of the issues around one group having an operation and the other not are removed.

Do you have any ideas on how to conceal the presence of a foreskin from the owner?

This is naive. Experimenter and experimentee effects apply all over the place. In other studies (such as ESP work) it is well known that innocent (and not-so-innocent) mistakes tend to be made in the direction the experimenter wants the experiment to go, and these experimenters (and the subjects) make no secret of their desire for circumcision to be protective.

I'm not being naive, I'm asking you to be specific. For example, the outcome measure was based on HIV serology. How would participant knowledge of whether they had a foreskin affect the results of HIV serology testing?

The circumcised group was given additional safe-sex promotion that the control group was not.

Both groups were given the same safe-sex promotion.

I need more detail about that.

Statistical methods are employed in epidemiology in order to (attempt to) make groups comparable. The sorts of geographical comparisons you made in your examples were too gross to draw any conclusions, while the studies you criticized used more specific measurements and comparisons in order draw conclusions that were more likely to be reliable and valid.

As above, the promotion was not identical for the two groups (as it would have to have been if they could have been blinded).

Since both groups were given the same promotion, can you clarify what you mean by this?

Circumcision can only be effective where safe sex is not practiced, ie where condom-promotion is ineffective. The more effectively condoms are promoted, the less effective circumcision must be. Wear a condom and the foreskin becomes moot.

The problem is that the measured effectiveness of condom promotion was about 20% or less in these studies.

Well that surprises me, and it suggests that aspirin is being overhyped as protective against heart attack, and the only reason it is promoted is that it is so proverbially harmless. I went to that site, clicked on "infectious diseases" and for treatments of HIV/AIDS, the NNTs were all under 21 and most in single digits.

This is expected. Examples of secondary and tertiary prevention should have lower NNT's than those for primary prevention. Even so, the same adjustment I mentioned earlier would apply since HIV/AIDS therapies are ongoing.

I'm not sure if that is statisically accurate. It assumes that the protection is uniform over time, and if you look at the charts of actual cases offered in the original papers, the numbers are so small that it's impossible to say if it is.

It is over-simplified in order to help make it clear. However, there is no particular reason to assume the protection varies substantially over time since prior research shows a similar effect in those men whose circumcisions were decades old.

Humans are not laboratory rats, and it may well be impossible to perform the kind of scientific experiment on them that one would like. One can't for example control when and where and with whom they have sex. It seems to me that these experiments were carried out with the same assumptions that can be used on lab animals, and the same confidence is now being placed on the results.

I understand that inexperienced and inexpert readers may make that mistake. However, experienced researchers and clinicians (such as the ones performing the study and using the results) are familiar with the differences between animal, efficacy and effectiveness studies.

There is also the Hawthorne effect, where behaviour changes just by the fact of being involved in an experiment.

Hence the point of randomization.

The other unfactored-in aspect is the extraordinary cultural role of circumcision, both for the subjects and for the experimenters. This, I suggest, greatly magnifies the importance of experimenter and experimentee effects.

Hence the point of randomization.

Linda

 

[Ivor the Engineer]

How does randomisation impact on the Hawthorne effect?

The Hawthorne effect is a form of reactivity, and describes a temporary change to behavior or performance in response to a change in the environmental conditions, with the response being typically an improvement.

 

[fls]

How does randomisation impact on the Hawthorne effect?

It doesn't.

Linda

 

[Ivor the Engineer]

It doesn't.

Linda

So why did you reply:

There is also the Hawthorne effect, where behaviour changes just by the fact of being involved in an experiment.

Hence the point of randomization.

?

 

[fls]

So why did you reply:

Randomization helps to distribute the various factors that may influence the outcome (such as the Hawthorne effect and other experimenter and experimentee effects)...well...randomly rather than systematically, so as to avoid assigning a bias to one group or another.

Linda

 

[Ivor the Engineer]

Which would be great, except for the fact the two groups did not have similar experiences.

ETA: Taxi's here. See you in two weeks.

 

[fls]

Which would be great, except for the fact the two groups did not have similar experiences.

That's kinda the point, isn't it?

Linda