Overreaching causal language in the social sciences

Overreaching causal language in the social sciences Across the social sciences, many studies use cross-sectional designs that reveal associations but are generally unable to support direct causal claims, yet authors of such articles may make or imply causal claims anyway. Here, to examine the prevalence of such ‘overreaching’ causal language, we analysed 194,631 cross-sectional articles using large language models. Over the period 1980–2024, an average of 46% of articles contained causal language in their titles or abstracts, where the annual rate has risen almost threefold since 2000 from 20% to 60%. To examine the effects of such language, we conducted a human-subjects experiment (N = 1, 105), finding that readers frequently indicate abstracts with this phrasing provide causal evidence but that methodological labels (β = −0.4, 95% confidence interval −0.56 to −0.19) and associational wording (β = −0.3, 95% confidence interval −0.43 to −0.07) reduce this tendency. Experiments with five LLMs revealed that model summaries of these articles (N = 100 each) can amplify causal overstatement, removing hedges and introducing causal claims where articles used strictly associational phrasing; however, prompting caution diminishes this pattern. That is from a recent paper by Calvin Isch, Timothy Dörr, Neil Fasching, Grace Jennings & Duncan J. Watts. Note that Isch is on the job market this year, working with Tetlock and Watts.