science fraud is common place

More than that - what if a single replication attempt doesn't, well, replicate? Is that a failure in whose experiment? Another replication required, etc. Can we at least check that the data in the first experiment was actually the data collected? That would save some replicating. And it's a really, really, basic quality control.
 
I was not really interested in the fraudulent paper kerfluffle. I am more interested in this
This thread is about fraudulent papers. So you are interested in derailing the thread :eek:!
The simple fact is that the detection of fraudulent papers gives a lower limit and estimates of the rate of detection gives estimates of the number of fraudulent papers. Assume it is 100% and ~0.01% of papers are fraudulent.

You are ignoring the real world where scientists are human, trained to be honest (especially in the medical fields), trained in environments where fraud is the deadliest sin and so are not comparable to the general public.
 
I know that is a claim made by science. I don't know that it's an absolute practice..
It is a claim made by scientists who know that it is not a absolute practice especially since there are circumstances where it cannot be done (how many universes can we experiment on :)).

Data being proprietary is irrelevant - we are talking about published results not proprietary results.
 
This thread is about fraudulent papers. So you are interested in derailing the thread :eek:!
The simple fact is that the detection of fraudulent papers gives a lower limit and estimates of the rate of detection gives estimates of the number of fraudulent papers. Assume it is 100% and ~0.01% of papers are fraudulent.
Hmmm and where does that estimate of the detection rate come from?

You are ignoring the real world where scientists are human, trained to be honest (especially in the medical fields), trained in environments where fraud is the deadliest sin and so are not comparable to the general public.

The telling point there is medicine I think. Medicine has been through a great re-vamp in the last few decades, and the results have been spectacular. But that doesn't apply to other research fields. Worse, I'm coming to feel that the rigour applied to medicine has made other researchers think of it as a 'technical' field (like engineering, as someone has said above) rather than science. ie, you can only 'do science' if you're free like a blue-sky researching bird, unfettered by all that messy 'proving' stuff.
 
My position - such as it is - is that we don't know what the level of fraud in research is.
Wow - a position that everyone in this thread has, MCH :eye-poppi!

In fact I would say that everyone in the world would agree that we do not know what the level of fraud in research is. But they would also say that we can estimate it (roughly!) as close to 0.01% in biomedical papers since that is the level of detected fraud and the detection should be extremely good given the scientific process (get a result, check it many times, never stop checking it).
 
I just don't think that the 'rigour and scrutiny' that is common to research is sufficient to label most research as 'scientific'.
The word scientific has little to with 'rigour and scrutiny'. Where is the scientific law that states that every experiment that does not have at least 2 supervisors will give the wrong results :rolleyes:?

It is the practice of science where 'rigour and scrutiny' comes in.
 
Wow - a position that everyone in this thread has, MCH :eye-poppi!
Well hurrumph, do you see why I was getting a little pished that it was being turned into something else?

In fact I would say that everyone in the world would agree that we do not know what the level of fraud in research is. But they would also say that we can estimate it (roughly!) as close to 0.01% in biomedical papers since that is the level of detected fraud and the detection should be extremely good given the scientific process (get a result, check it many times, never stop checking it).
Well that's what needs some scientific testing. Is the detection really that good? ie, are results checked many times? How many? How long does it take (ie, how many fraud papers are extant at any one time?)

Even if you can make that claim in biomedical (and that's quite a claim) what about 'science in general'?
 
The word scientific has little to with 'rigour and scrutiny'. Where is the scientific law that states that every experiment that does not have at least 2 supervisors will give the wrong results :rolleyes:?

It is the practice of science where 'rigour and scrutiny' comes in.
What is this practice that brings scrutiny without scrutiny?!
 
Hmmm but I think that's an assumption that doesn't actually happen in practice.
Hmmm, MCH, replication actually happens in practice - ask any physics student replicating Newton's Laws! For that matter if you have done any science in school, MCH, then you have replicated earlier experiments :D.

Actually we do have many way of telling when results are 'generally accepted'. Citations are acceptance. Inclusion in textbooks are acceptance. Comments by scientists are acceptance. Appearances in popular science books are acceptance. Even news articles can be acceptance.

Citations are an indication of replication since papers that replicate experiments kind of cite the original experiments :jaw-dropp!
 
Hmmm, MCH, replication actually happens in practice - ask any physics student replicating Newton's Laws! For that matter if you have done any science in school, MCH, then you have replicated earlier experiments :D.

Actually we do have many way of telling when results are 'generally accepted'. Citations are acceptance. Inclusion in textbooks are acceptance. Comments by scientists are acceptance. Appearances in popular science books are acceptance. Even news articles can be acceptance.

Citations are an indication of replication since papers that replicate experiments kind of cite the original experiments :jaw-dropp!
There's plenty of statements upthread about how replications don't happen: funding, willingness, interest, politics, etc. Newtons Laws are very very old and readily replicated and not a good example of modern concepts! Who has an LHC to hand to replicate the discovery of the Higgs Boson?

Citations can be for replication, yes, but also for other reasons. What is the rate for citation for replication vs citation for refutation?
 
...Especially the small inspections: did this data actually come from the instrument?
The small inspections that demand that thousands (millions) of people be hired to watch every experiment that is conducted in the world :rolleyes:?

In real world, scientists start with a tiny bit of trust. If someone reports that data comes from an instrument then they partially trust that the data come from that instrument. Subject to their expertise though - it may be easy to see that that instrument could not produce that data.
Then scientists throw away trust and see if the data can really come from that instrument (replication of results).
 
Would that be like priests ...?
No. It would be like doctors going to a medical school, learning about ethics, learning the rules of clinical trials, swearing an oath, learning what a dishonor it would be to falsify results, etc.
The physical sciences skip the people oriented bits and oaths.
 
yes I should have been more specific - I did say this later above - but quality controls of a paper are not quality controls of the work. It's too late.
It depend on what you mean by "quality controls", MCH.
It is certainly too late for the quality controls in a factory after the product rolls out of the door. But science is more like a factory with a circular production line. The product never rolls out of the door. Quality control never stops.

If the results in a paper are not replicated then in theory it should be in limbo awaiting verification. Unfortunately as you have seen this is sometimes not the practice.
 
Last edited:
If you don't have that basic inspection - that the instrument did indeed produce those results - then you don't have that most basic of quality controls. Everything from there is based on trust, and we're back to how much we can trust ordinary people. Or some evidence that researchers are not ordinary. Or some 'gossip' model that eventually the truth will out. Eventually!

Even when replicating - if it happens - who has done it? Are they people associated with the original people? What influences are in that community?

I understand there are practical issues (I deal with them, many industries do), but just because it's not done because it's hard is no reason to then claim that the work is sound and scientific.

We, quite rightly, don't trust people - we trust inspected people. With the right inspection.
 
No. It would be like doctors going to a medical school, learning about ethics, learning the rules of clinical trials, swearing an oath, learning what a dishonor it would be to falsify results, etc.
And even then, we have double-blinded trials because all that stuff isn't enough
 
More than that - what if a single replication attempt doesn't, well, replicate?
It is basically up to the authors of the first paper to show that the second paper made some errors.

You may not know this but the raw data for most papers is available to anyone who asks for it. In some cases there may be reasons why it cannot be released, e.g. patient confidentiality. In some cases there may be practical problems in obtaining it, e.g. the LHC produces terabytes of raw data. On the other hand, many authors make the raw data automatically available over the Internet. This is especially true in astronomy.
 
Hmmm and where does that estimate of the detection rate come from?
Hmmmm it is not an estimation - it is an assumption:
"Assume it is 100% and ~0.01% of papers are fraudulent."

But that doesn't apply to other research fields.
Well that would be because those fields are not medicine!
Can you imagine the absurdity of having someone ticking off every byte in the LHC data?
What about having someone stationed on the Hubble telescope just to make sure that the Hubble data comes from the Hubble?

As someone with a physics background, I would argue that physics has been the rigorous and scrutinized research field and medicine is just catching up :rolleyes:.
 
It is basically up to the authors of the first paper to show that the second paper made some errors.
So the discourse carries on within the community, which is fine. Doesn't really answer the question about how replication is supposed to be a quality control.

You may not know this but the raw data for most papers is available to anyone who asks for it. In some cases there may be reasons why it cannot be released, e.g. patient confidentiality. In some cases there may be practical problems in obtaining it, e.g. the LHC produces terabytes of raw data. On the other hand, many authors make the raw data automatically available over the Internet. This is especially true in astronomy.
'Most papers'? It's really not, at least not routinely. What about code? Again, not routinely. Even in astronomy - have you tried talking through the settings used on SExtractor for any particular research paper?! Or derived dataset?!
 
If you don't have that basic inspection
See my (a bit sarcastic) remark about the absurdity of that basic inspection when applied to the Hubble telescope.
Can you see why it would be absurd to demand that the various Mars rovers be inspected to make use that the data actually comes from them :rolleyes:?

The fact remains that this "basic inspection" is not needed and is impossible in many situation.
 

ISF - Join now!

Every member here is approved by hand. No bots, no spam, just people who care about evidence and honest debate.

Membership is free!

Create your free account

Back
Top Bottom