The controversy around hoax studies in critical theory, explained

An argument for men self-penetrating with dildos to reduce transphobia. An ethnography of men who attend “breastaurants” like Hooters. Research on rape culture among the dogs at Portland dog parks.

These are all real articles, published in real, scholarly journals written by a group of three people — magazine editor Helen Pluckrose, mathematician James Lindsay, and philosopher Peter Boghossian. But if they sound to you like parodies of a certain style of academic research, you’re right. Pluckrose, Lindsay, and Boghossian wrote and published the articles as part of year-long hoax campaign targeting fields like gender studies.

Pluckrose et al. wrote 20 papers and submitted them, under false names, to a variety of journals. By the time they ended the experiment in early October, seven out of the 20 had been accepted for publication.

In a lengthy write-up explaining the sting, the authors describe their hoax as proof that fields focusing on identity — gender studies, queer studies, critical race studies, etc. — were “corrupt” to their core. “Grievance studies,” as they chose to refer to these fields, elevate politically fashionable nonsense over rigorous scholarship; Pluckrose et al. see them as a cancer on the university that needs to be excised.

“These fields of study do not continue the important and noble liberal work of the civil rights movements,” they write. “They corrupt it while trading upon their good names to keep pushing a kind of social snake oil onto a public that keeps getting sicker.”

This isn’t the first time that academic journals have been pranked to prove a point. But this time, the prank has resulted in backlash against the pranksters themselves. Some scholars from fields unrelated to so-called “grievance studies” have come out swinging at the prank, rejecting Pluckrose et al.’s methods as unscientific and the conclusions they draw about the entire field of identity studies unsupported by their evidence.

“God this is dumb,” Harvard political scientist Matt Blackwell tweeted, referring to part of the Pluckrose et al. write-up. “It’s an amazing self-own that these people didn’t even engage in a scientific process.”

Both sides have a bit of a point, though I think the critics get the best of it.

The hoaxers are right that there are problems in identity studies, and that one of those problems is political bias. But their experiment is not convincing evidence that these problems are necessarily worse or more fundamental than those that affect other fields, including ones that seem more “scientific” like psychology or economics.

What’s really at stake here is less a dispute over academic method and more an opening of another front in America’s great culture war. Pluckrose, Lindsay, and Boghossian are not neutral scholars: All of them are critics of so-called identity politics and the social justice left more broadly. When I spoke to Lindsay on the phone, he told me openly that the project was born of a concern over “political correctness” run amok in the academy and the United States more broadly.

Their goal, in short, is to reveal the identity left as an emperor wearing no clothes: To show that the ideas you hear from liberal intellectuals, activists in #MeToo and Black Lives Matter, and even some Democratic elected officials are vacuous at best.

Pluckrose et al. see this as a war to save liberalism from itself. But it actually reveals a shift in modern political debate, one in which ideology and identity have supplanted some of the old left/right divisions — aligning a certain brand of liberal with the forces of reaction.

The Grievance Study hoax started with concerns that people were being unfairly accused of racism and sexism — illustrating the political stakes of all of this practically from the get-go.

Boghossian (a professor at Portland State University) and Lindsay (who has a PhD in mathematics and works outside the academy) were both involved in the atheist-skeptic writing community. In recent years, they had both been alarmed by what they saw as a wave of unfair criticism directed at individuals under the banner of social justice and “systemic” discrimination.

One such example, per Lindsay, is James Damore, the Google engineer fired in 2017 after writing a memo defending the notion that men are intrinsically better suited to the technology field than women.

“We noticed a trend that included people we respected being pilloried, in several cases, on accusations of racism and sexism,” Lindsay tells me. “When we looked into these more closely, we realized that they were using particular definitions of sexism and racism — specifically, they were using ‘systemic’ racism and sexism.”

The theory of “systemic” discrimination holds that the issue is not merely a matter of individual attitudes, but rather of broader social structures and ideas. The American criminal justice system is racist under this theory because, despite formally guaranteeing equal protection, it is set up in such a way that black people end up getting arrested and incarcerated at wildly disproportionate rates.

In Lindsay and Boghossian’s view, the notion of systemic racism and sexism is used to attack people for all sorts of behaviors that are not motivated by personal racial animus. They blame a particular application of poststructural theory that’s become popular in fields like gender studies, one that focuses heavily on the role of language in maintaining oppressive structures.

“Political correctness is one public manifestation of a strong focus on the idea that language constructs society through establishing and maintaining power imbalances, which is a very [poststructural] idea,” Lindsay says.

The hoax idea was conceived of as a way to test the quality of the underlying research behind this theory. If poststructural theory was rooted in rigorous academic research, then its scholars should be able to ferret out fake papers during peer review. But if they can cover a totally nonsensical argument in poststructural jargon, and get it published, that would suggest the entire edifice is corrupt.

Lindsay and Boghossian’s first attempt didn’t go so well. In 2017, they published a paper called “The Conceptual Penis as a Social Construct” in the journal Cogent Social Sciences. It argued that penises were better understood as a kind of masculine identity than a biological organ, and that the “conceptual penis” was responsible for, among other things, climate change.

The problem, as critics pointed out, is that Cogent Social Sciences was not a leading gender studies journal, or even a reputable academic outlet at all. It’s an extremely low-quality publication that charges a $625 fee to publish. Indeed, when Lindsay and Boghossian tried to submit “The Conceptual Penis” to an actual journal of gender studies, NORMA, it was rejected.

For these reasons, Lindsay told me, the experiment “mostly failed.” The grievance studies hoax was designed explicitly in an effort to address the shortcomings of the conceptual penis stunt. They brought on Helen Pluckrose, editor of the magazine Areo, as a contributor; they also improved their methodology.

“We made sure, for instance, we’d only use peer-reviewed journals. We’d use the highest-impact factor journals within the aims and scope of the kinds of paper we were writing,” Lindsay explains. “We would never use a pay-to-publish journal ... and we would also do more than one paper.”

At first, their papers were rejected. The content was too obviously silly, too much of a joke. But then they committed to studying the kinds of papers that had been written on the topic in the past, and improved their ability to mimic the arguments in them. This was a full-time job for Lindsay: He secured funding from a group of donors whose names he would not reveal to spend, in his words, “90 hours a week” on this project.

The result was that by the end, seven out of the 20 papers were accepted for publication in target journals. As Slate’s Daniel Engber notes, three of the seven are the most compellingly absurd examples and the ones that have received the most attention: the dildo, dog park, and breastaurant papers I highlighted earlier.

These three accepted papers contain made-up research, like 10,000 hours of fake observation of dogs humping other dogs at dog parks. They also are filled with sentences like “because of my own situatedness as a human, rather than as a dog, I recognize my limitations in being able to determine when an incidence of dog humping qualifies as rape.”

And the papers come to some strange conclusions. The dildo paper finds, on the basis of “semi-structured interviews with thirteen men,” that there’s “potential socially remedial value for encouraging male anal eroticism with sex toys.” Put more simply, the paper argues that men who masturbate by penetrating themselves anally might be less homophobic and transphobic as a result.

The fact that these silly-sounding arguments were accepted at all, according to Lindsay, vindicates their initial assumption: that much of poststructural theory underpinning modern arguments about race and gender is rotten, if not entirely vacuous.

“If we’re going to base policy on those things, and activism on that research, I think that the scholarship itself has a duty to the public to be trustworthy,” he says. “I’ve lost my ability to trust. ... I’m sure some of the work is very good. The problem is now I don’t know how to tell.”

At times, Lindsay went even further. If the various identity studies subfields can’t be reformed, he told me, then those departments — some of the relatively few in which women and minorities have a significant presence — should be shut down.

“These departments can continue to do what they want to do, but outside of the university system,” he told me.

The reaction to the Grievance Study hoax was mixed. Some scholars, like Harvard lecturer Yascha Mounk, celebrated their findings. Others were more critical. James Stacey Taylor, a libertarian philosophy professor at the College of New Jersey, pointed out that two of the journals they submitted to — Afflia and the Journal of Poetry Therapy — “aren’t really academic outlets at all but seem aimed at practitioners,” meaning social workers and therapists.

It seems clear the hoaxers exposed a problem with these journals. Academic identity studies is committed to a progressive political view; in fact, it’s more accurately described as a radical one. The hoax experiment shows that it’s possible for bad work to be published that flatters those biases, which is certainly something reviewers working in the field’s journals should be more wary of than they seem to be.

At the same time, though, any field can be hoaxed if you lie about the data you’ve gathered and hide your true intentions: The entire system depends on good faith and honesty. When people break it, by submitting bad-faith arguments or by manufacturing data, the system is not well-equipped to catch it.

My colleagues Brian Resnick and Julia Belluz, for example, have done extensive reporting on problems with statistical research in psychology and health, respectively. A shockingly large percentage of papers, even in leading journals, can’t be replicated in follow-up experiments. Part of the problem is researchers selectively reporting the results of their research to make their conclusions seem rigorous when they aren’t, a trick enabled by a kind of a statistical cheat called p-hacking.

Peer reviewers can’t tell if a statistical paper is flawed in this way, because by definition they don’t have access to research results that are excluded from the paper. They have to trust that the researchers came by their conclusions honestly; the entire peer review system depends on establishing a certain level of trust.

This is why it matters, as Engber notes, that the most striking papers from the hoax actually invented interviews and data from observation. “We know from long experience that expert peer review offers close to no protection against outright data fraud,” he writes. “These examples haven’t hoodwinked anyone with sophistry or satire but with a simple fabrication of results.”

In order to draw the big ideological conclusions about gender and identity studies that Pluckrose et al. want to, you need to show that this is something different from the standard problems with academic publishing.

But the hoax doesn’t show that fake papers are more likely to be accepted than real papers, nor does it show that gender studies and journals of poststructural theory are more likely to accept fake papers than those in any other field. Without this kind of comparison, it’s hard to know if gender studies is uniquely corrupt — or if there’s a bigger flaw in the peer review system writ large.

Lindsay, in our conversation, expressed exacerbation at this line of reasoning. For one thing, it would have been logistically impossible for his team to learn all sorts of different academic disciplines and submit to journals in them. Moreover, he argues, it’s irrelevant: The fact that there might be problems in other fields doesn’t mean there isn’t a problem with research on identity topics.

“Suppose the ATF runs a big sting on a nightclub that they think is running a drug dealing operation ... and they go undercover, run a sting operation, and they bust them,” he says. “Nobody calls the ATF and says, ‘Why didn’t you check out grocery stores to see if they’re selling beer to minors?’ You didn’t check the Safeway to see if it had similar problems.”

But drug sales at a nightclub have nothing to do with alcohol sales at a grocery store, whereas academic publishing has certain relevant similarities and weaknesses across fields. In order to show that the ideas underlying identity studies are corrupt relative to the ideas that dominate in the more traditional sciences, you have to show that those ideas make their journals more manipulable than those of other fields are. And that doesn’t appear to be the case.

Cornell University food researcher Brian Wansink, for example, built his career on studies arguing that people’s psychology and environment subtly but profoundly work to make them overweight. But it turned out his work is dubious, relying heavily on what one expert called “p-hacking on steroids.” Wansink has announced he is resigning from his post at Cornell next year.

No one would say Wansink’s downfall discredits the very idea that the environment affects our weight — just that his specific claims can’t be supported based on the available evidence. By the same token, the Grievance Studies hoax doesn’t discredit the idea of “systemic” racism or sexism; all it’s shown is that work that depends on related ideas can be bad and still get published.

Pluckrose et al. have failed to show that qualitative inquiry informed by poststructural theory is particularly prone to being manipulated, or that the core claims in the various identity studies disciplines are invalid.

“We already know all kinds of crap is published because people are willing to put a thumb on the scales,” Duke sociologist Kieran Healy tweeted. “If you hate an area enough you can gin up a fake paper and get it published somewhere if you try. The question is, what do you hate?”

To understand the answer to Healy’s question, you really need to understand the academy of the 1990s.

Similar to today, the academy of the 1990s was obsessed with debates over “identity politics” and “political correctness.” In the world of academic research, this manifested in something called the “science wars”: a debate, to oversimplify greatly, about whether scientific research really establishes “objective truths” about the world.

Natural scientists and their allies in disciplines like philosophy defended the traditional idea of objectivity, while critics in fields like science and technology studies argued that scientists were ignoring the inherent limitations on objectivity created by human perspective and biases. Many of the critics were concerned about the way that power structures, like racism and sexism, had historically twisted scientific inquiry (eugenics and Nazi race science, for example).

In a prank that would inspire Pluckrose et al. years later, NYU physicist Alan Sokal submitted an article to the poststructural journal Social Text titled “Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity.” The article made a series of obviously absurd claims, including that gravity itself was a social phenomenon, dressed up in the language of poststructural philosophy. The point was to show that you could get away with anything if you made it sound properly poststructural and come to the right political conclusions.

“The results of my little experiment demonstrate, at the very least, that some fashionable sectors of the American academic Left have been getting intellectually lazy,” Sokal wrote in an article revealing his hoax. “The editors of Social Text liked my article because they liked its conclusion: that ‘the content and methodology of postmodern science provide powerful intellectual support for the progressive political project.’”

Sokal’s hoax was self-consciously more limited than the Pluckrose et al. sting (which some observers are calling “Sokal Squared”). But it reveals what’s going on here: These hoaxes are not so much about academic hygiene as they are about discrediting one’s political opponents.

Now, all three of the Grievance Study authors identify as political liberals. Lindsay emphasized this point in our phone call, noting his personal support for significantly higher marginal tax rates on the wealthy.

But they are liberals of a very particular kind: ones who are vehemently opposed to the current wave of left-liberal organizing surrounding race and gender. Pluckrose wrote a piece in 2016 titled “Why I No Longer Identify as a Feminist,” arguing that “western feminism needs to stop focusing on ‘trivial bullshit,” and that “I don’t have a huge amount of sympathy for women who feel traumatized and excluded by scientists’ shirts or video games.”

In September of this year, Pluckrose and Lindsay co-authored a piece blasting practitioners of “identity politics” as betraying the liberal legacy of the civil rights movement, polarizing the American mainstream against progressive causes. “Identity politics in the form of Social Justice ... could undo decades of social progress and provide a rationale for a resurgence of racism, sexism, and homophobia,” they write.

This kind of view is on its face progressive, but is in fact most commonly heard today from moderate conservatives and members of the so-called “Intellectual Dark Web.” The argument is that today’s activists, including some of the most prominent #MeToo and Black Lives Matter advocates, are alienating white men through their confrontational language and tactics.

This leads to some odd alliances. Boghossian, for example, is a longtime collaborator of Stefan Molyneux, a YouTube personality and alleged “cult leader” best known for his belief that “blacks are collectively less intelligent.” Boghossian is a frequent conversation partner of Molyneux’s and even wrote the forward to one of his books. In an email, Boghossian told me this was because the two of them have “congruent beliefs about metaphysics” — not believing in God or the supernatural, specifically — and that they disagreed on politics.

But this isn’t the full picture. In one video, for example, Boghossian and Molyneux take turns attacking modern feminists and social justice advocates more broadly. “The left are the new racists,” Boghossian says 13 minutes into the conversation:

This all goes to show that the ideological battle lines in modern politics are shifting. Conflict does not fall so neatly along policy lines, where people who agree on tax levels or same-sex marriage agree with each other. This is the way things mostly were for the past decade of American politics, and something we’ve gotten pretty used to.

Rather, things are looking a bit more like the 1990s. Our rhetorical battles today are over how people identify themselves, the kinds of arguments they make, and how they feel about particular movements for social justice. You can be all for same-sex marriage as a legal institution, for example, and still be deeply opposed to the way the modern social justice left thinks and talks. That puts you on the same side of the argument as the right, even if you frame yourself as a liberal in doing so.

Lindsay rejected this argument in our conversation. “About being a tool for the right: Have you seen me go on Tucker Carlson yet? Do you think he hasn’t asked?” he told me, rhetorically.

Fair enough: He’s self-conscious enough to know what doing that particular television show would entail. But the fact that this is the type of audience that’s excited about the Grievance Studies hoax says a lot about whose work the project is actually doing.