Hi,
Chris
, as for your question:
"So here is my question and anyone on either side is welcome to answer: when a scientific paper is beinmg published to test the findings of a prior study, is it expected that every aspect of the original protocol be followed to the letter, even if parts of the original study are believed to have been poorly carried out? Or is it more standard to ask the same question and then replicate only the relevant parts, and also filling in the missing parts of the original study (which is what Millette did)?
I don't want just the standard debunker answer, I'd like to know what the tradition is in scientific circles when publishing a paper that follows up on a prior published work."
Personally, I do not remember any scientific paper, which was replicated "verbatim". But, some of its parts are replicated frequently for various reaons.
My job for ca 25 years has been a multistep synthesis of novel complex
(conjugated) polymers and their monomers and precursors
(I have ca 60 per-reviewed papers in the field). In this regard, my work is kind of "pioneering" one
(I do not replicate anybody's whole work), but my new polymers are frequently similar to others, already known, and some precursors are frequently either similar or the same as in published papers.
As any organic/polymer synthetic chemist knows pretty well, replications of published synthetic "recipes" frequently do not work well
(low yields, no yields, too complex reaction mixture as a result, etc.), so one is forced to find reasons and solve the problem in some different manner. It can be later described in the paper e.g. like
"stuff xy was prepared using this variation of original method" or so. Important is the result, i.e. the well-defined polymer with those and those characterictics and analytical data.
As for analyses, chemists usually know well, what is necessary for proper characterization of the stuff and what can be replaced with what
(which means a "non verbatim" characterization in comparison with original paper). E.g., in the case of low molar mass precursors, traditional duo "nuclear magnetic resonance - elemental analysis" can be with advantage replaced with the tandem "mass spectrometry - gas chromatography". Indeed, both such results are accepted by any qualified chemist.
There is some basic set of analytical methods which are typical/necessary for characterization of some "groups of compounds".
E.g., for soluble organic polymers, the necessity is to measure their distribution of molar masses (lengths of polymer chains) using GPC, light scattering etc.
On the other hand, there is e.g. no reason to check if prepared stuff is polymer
(or some composite of prevailing polymer and incombustible fillers, like some paint) by burning it at high temperatures, since this is
self-evident. Any organic polymer simply
burns/is combusted at high temperatures, giving some heat, e.g. at 430 degrees C and similarly high temperatures..
For the replication of any part of any paper, the synthetic chemist must first survive, therefore he has to know the expected behavior of tens of materials and hundreds of common chemicals. Similarly, he has to have some kind of "intuition"/experience like
"this product has this property (very typically, thin layer chromatography spot on the expected place), hence it is/it is not perhaps the proper one".
Here, the comparison with some other stuffs is handy, but such comparison should be accompanied with some basic knowledge/common sense. E.g., the comparison of liquid product with some other accidentally chosen and unspecified liquid is an apparent idiocy, since they are millions of various liquid chemicals known to chemists.
To give you another example, the comparison of some sample which can be paint
(or something else) with some accidentally chosen (and not specified) paint(s) is the apparent idiocy as well, since they are perhaps even hundreds of thousands of various paints available with different properties. This is anyway the reason, why we do not have only one, two or three paints.
Apparently, such kind comparison used in any paper is worthless and does not need replication. What is even worse, any replication cannot be done, since materials used for comparison are unspecified. But, I have never met any such bad comparison in the per-reviewed papers on polymers.
But, back to your question. I'm an occasional reviewer and some of the papers I have reviewed were really so bad, that I was forced to refute them. As for reading per-reviewed papers, they usually show that these chemists know well what they are talking about and use only proper/relevant methods for the study. Ocassionaly, I've asked: Well, I have repeated everything 30 times without success, isn't some mistake on the side of recipe authors? And occasionally, I'm right
(I know well that my answer is little bit "biased" and diverted from your interest, Chris, but the temptation was too strong)