Wanky research pulled off website

[Elaedith]

I think people who love to bring up the Sokal Paper miss two important things:

1. It was published in a journal that did NOT (yet) do any peer review, so it really doesn't prove much. It's like saying I could self publish something written by my simple Markov Chains program on Amazon. Yes, I could. So what exactly does it prove about anything else? I mean, it's not a whole lot different than posting it on this site, which as the link above proves, I jolly well could.

2. It was in a journal of postmodernist twaddle, and frankly, it wasn't all even that ill-fitting there. I would assume that even after being told it's a hoax, a lot of post-modern philosophy professors were left scratching their heads as to how could you tell it apart from a genuine one. Or would be, if they did any honest introspection


Basically a lot of people seem to imagine it shows some profound and lamentable stuff about the state of peer-reviewed technical journals, when the whole point is that it wasn't either. If anything Sokal just showed what happens when you DON'T do peer review.

I don't think that's entirely accurate. The journal did not submit Sokal's paper to external peer review by a physicist, but it had editorial staff who reviewed papers and made publishing decisions. Neither of the editors suspected that Sokal's paper was a parody. So it partly illustrates the dangers of not using external peer review, but also that the editors could spot nonsense when it affirms their ideological position.

 

[JoeMorgue]

Also the Sokal Paper wasn't submitted to like... the New England Journal of Medicine or the Lancet but a self described "Post Modern Cultural Study" academic journal, which is about as soft as science gets. Hell at journal in question didn't even practice peer review, so the whole affair is like someone proving door locks don't work by opening a door that doesn't have a lock on it.

 

[SpuriusMaximus]

I don't think that's entirely accurate. The journal did not submit Sokal's paper to external peer review by a physicist, but it had editorial staff who reviewed papers and made publishing decisions. Neither of the editors suspected that Sokal's paper was a parody. So it partly illustrates the dangers of not using external peer review, but also that the editors could spot nonsense when it affirms their ideological position.

Staff qualified in what exactly? Right, in the postmodern school subset of the philosophy of language, since that's what the focus of the journal was. And I'm telling you that you don't even need any ideology (except in as much as it's inherent in postmodernism) for that paper to read like a perfectly normal one.

I mean, what would you even check there? Any branch where you can test your claims against reality is now filed under science, not philosophy. Even when you talk about how a culture or social group thinks about something, that tends to fall under sociology or anthropology if you actually want it to have hard reality checks. A philosophy paper is just a guy thinking really hard about stuff, but there is no actual check you can do other than whether their quotes from dead bearded guys are true, and yes, whether it fits the theme of that school of philosophy. But otherwise, unless, say, he cites Descartes as saying "don't trust every quote on the internet" and you determine that Descartes wrote no such thing, there is no reality check you CAN do in that field.

Most that can happen is that some other respectable philosopher writes his own paper or book about why the first guy was wrong.


Now unlike, say, our esteemed coleague JoeMorgue, I'm not against philosophy per se. I'm not gonna discourage anyone from thinking hard about anything. Even if it's at the level of Beavis and Butthead's "Why is it called taking a dump when you're not taking it anywhere" (which is actually fitting here, since it IS philosophy of the language), it still beats not thinking hard about anything, as far as I'm concerned.

But I also wish more people understood that fundamental difference between philosophy and science. And thus the difference between what passes for a paper, journal or peer review in each domain.


TL;DR: you really can't extrapolate from what's publishable in a postmodern philosophy journal to what is publishable in a science journal, nor viceversa. What is perfectly normal content in the former can be an epic fail in the latter, and viceversa.

 

[d4m10n]

TL;DR: you really can't extrapolate from what's publishable in a postmodern philosophy journal to what is publishable in a science journal, nor viceversa. What is perfectly normal content in the former can be an epic fail in the latter, and viceversa.

What is considered unpublishable in a postmodern philosophy journal? If imposters can slip in a serious sounding paper arguing that social justice activists can righteously make fun of others but that it is problematic for others to make fun of them, I'm wondering where the line is drawn.

 

[Myriad]

Staff qualified in what exactly? Right, in the postmodern school subset of the philosophy of language, since that's what the focus of the journal was. And I'm telling you that you don't even need any ideology (except in as much as it's inherent in postmodernism) for that paper to read like a perfectly normal one.

I mean, what would you even check there? Any branch where you can test your claims against reality is now filed under science, not philosophy. Even when you talk about how a culture or social group thinks about something, that tends to fall under sociology or anthropology if you actually want it to have hard reality checks. A philosophy paper is just a guy thinking really hard about stuff, but there is no actual check you can do other than whether their quotes from dead bearded guys are true, and yes, whether it fits the theme of that school of philosophy. But otherwise, unless, say, he cites Descartes as saying "don't trust every quote on the internet" and you determine that Descartes wrote no such thing, there is no reality check you CAN do in that field.

Most that can happen is that some other respectable philosopher writes his own paper or book about why the first guy was wrong.


Now unlike, say, our esteemed coleague JoeMorgue, I'm not against philosophy per se. I'm not gonna discourage anyone from thinking hard about anything. Even if it's at the level of Beavis and Butthead's "Why is it called taking a dump when you're not taking it anywhere" (which is actually fitting here, since it IS philosophy of the language), it still beats not thinking hard about anything, as far as I'm concerned.

But I also wish more people understood that fundamental difference between philosophy and science. And thus the difference between what passes for a paper, journal or peer review in each domain.


TL;DR: you really can't extrapolate from what's publishable in a postmodern philosophy journal to what is publishable in a science journal, nor viceversa. What is perfectly normal content in the former can be an epic fail in the latter, and viceversa.

What the Sokal paper showed was that what had been presented to the outside world as a highly technical, specialized, and meaningful jargon was in fact indistinguishable from nonsense, even by the supposed insiders in the field. The transformative hermeneutics of quantum gravity described in the paper wasn't just scientifically invalid or inadequately supported by its sources or overly speculative, it was outright gibberish.

 

[SpuriusMaximus]

What is considered unpublishable in a postmodern philosophy journal?

Presumably something that actually makes sense

 

[theprestige]

I think people who love to bring up the Sokal Paper miss two important things:

1. It was published in a journal that did NOT (yet) do any peer review, and did NOT ask any expert's opinion before publishing it, so it really doesn't prove much. It's like saying I could self publish something written by my simple Markov Chains program on Amazon. Yes, I could. So what exactly does it prove about anything else?

I mean, it's not a whole lot different than posting it on this site, which as the link above proves, I jolly well could. Just that nobody stopped me from posting it in the religion and philosophy forum here, doesn't really say anything about the state of philosophy (or religion) anywhere else.

2. It was in a journal of postmodernist twaddle, and frankly, it wasn't all even that ill-fitting there. I would assume that even after being told it's a hoax, a lot of post-modern philosophy professors were left scratching their heads as to how could you tell it apart from a genuine one. Or would be, if they did any honest introspection But at any rate, I don't expect they actually had anyone qualified in Physics in their staff, nor I suspect were they interested in how the Physics part checks out, as long as it does the right kind of navel gazing.


Basically a lot of people seem to imagine it shows some profound and lamentable stuff about the state of peer-reviewed technical journals, when the whole point is that it wasn't either. If anything Sokal just showed what happens when you DON'T do peer review.

It's moot, since as d4m10n's tragicomic example illustrates, there's been worse since.

It turns out that far from plumbing the depths of the problem, Sokal only touched the tip of the iceberg. And over the years, the more we look at the problem, the worse it seems to be. There's a perverse incentive to not replicate experimental results. Bad research is being published in peer reviewed journals not merely by white hats seeking to expose the weakness of the system, but also by black hats seeking to exploit that weakness. Not to mention poor research being done by poor researchers, that also doesn't get filtered - the one basic job of peer review! And not to mention all the good researchers who fall into a pit of bad research because they trusted the peer review system.

And going back to the original exchange, it was suggested that the reasons this particular paper got published would remain a mystery. I don't think that's true. I think the understanding that we've been taking the journal system for granted, when it's actually open to abuse and rife with untrustworthy results masquerading as good science... I think that understanding has been with us since Sokal. Even though Sokal doesn't even really get into the full scope of the problem.

We've known about the problem since around that time. That's what I'm saying. The mystery alluded to has been solved.

 

[theprestige]

Presumably something that actually makes sense ; )

Now I want to try to publish a paper in a postmodern philosophy journal, that makes the sensical argument that publishing something sensical in a postmodern philosophy journal is the most nonsensical thing, and therefore the paper should be published. The implied paradox would be the cornerstone of my argument.

 

[SpuriusMaximus]

Err... why not? I mean, you even have the Sokal template for it. Just frame it as hermeneutics, and discuss the linguistic constructs and cultural expectations for the various journals. That they result in a paradox would probably actually increase the value of the paper.

Me, I'm more like wondering if I could get Mark V Shaney (you know, that statistics-building abomination I talked about earlier) to write a publishable paper. Actually, I'm fairly confident the 4 word chain version would produce something that at least sounds like (someone deranged) trying to tell you something deep, if I train it on several megabytes of existing philosophy papers.

 

[Lplus]

Err... why not? I mean, you even have the Sokal template for it. Just frame it as hermeneutics, and discuss the linguistic constructs and cultural expectations for the various journals. That they result in a paradox would probably actually increase the value of the paper.

Me, I'm more like wondering if I could get Mark V Shaney (you know, that statistics-building abomination I talked about earlier) to write a publishable paper. Actually, I'm fairly confident the 4 word chain version would produce something that at least sounds like (someone deranged) trying to tell you something deep, if I train it on several megabytes of existing philosophy papers.

For a second I read that as 4chan....

 

[Ziggurat]

Me, I'm more like wondering if I could get Mark V Shaney (you know, that statistics-building abomination I talked about earlier) to write a publishable paper. Actually, I'm fairly confident the 4 word chain version would produce something that at least sounds like (someone deranged) trying to tell you something deep, if I train it on several megabytes of existing philosophy papers.

There's a term that's useful here, both for computer-generated and human-generated content:

Pseudo-profound bull ****

 

[SpuriusMaximus]

Ah. Thanks. Now I have a more concise way of describing most meetings at work

 

[SpuriusMaximus]

For a second I read that as 4chan....

To be fair, I do suspect that some people ARE using Mark V Shaney type programs to write their nonsense on various online boards. I mean, Bikewer mentioned a Quora poster in my original Mark V Shaney thread.

But I can't prove that it's actually that or just run-of-the-mill brain damage. (A stroke is the most common cause of aphasia.) So take that suspicion of mine with a lot of salt.

 

[Hellbound]

There's a term that's useful here, both for computer-generated and human-generated content:

Pseudo-profound bull ****

I always called it “Yoda wisdom”. Give the appearance of being profound, while mostly passing in mystic nonsense and trite cliches.


Sent from my volcanic island lair using carrier pigeon.

 

[SpuriusMaximus]

I always called it “Yoda wisdom”. Give the appearance of being profound, while mostly passing in mystic nonsense and trite cliches.

While mystical stuff does tend to rely on this, it seems to me like it's much more widespread than that.

E.g., I swear to FSM I'm not making it up, I actually was reading the style guide of a corporation, which shall remain unnamed to protect the idio... err... innocents. And it mandated the use of a specific font, because it looks "sovereign" and a few other words that didn't even mean anything about a frikken' FONT. Didn't even look all that different from Callibri, to be honest. And then there were the sections on colours and rectangular buttons. Someone who obviously didn't even actually know the standard names for those colours, because every single name they used for an RGB triplet was the wrong one, felt a need to write several random pompous-sounding words about each that didn't even mean anything.