Does the public know much more about politics than conventionally thought? A number of studies have recently argued, on various grounds, that the “don’t know” (DK) and incorrect responses to traditionally designed and scored survey knowledge items conceal a good deal of knowledge. This paper examines these claims, focusing on the prominent and influential argument that discouraging DKs would reveal a substantially more knowledgeable public. Using two experimental surveys with national random samples, we show that discouraging DKs does little to affect our picture of how much the public knows about politics. For closed-ended items, the increase in correct responses is large but mainly illusory. For open-ended items, it is genuine but minor. We close by examining the other recent evidence for a substantially more knowledgeable public, showing that it too holds little water.

Published version
Appendices
Replication materials

[BibTeX] [EndNote] [Google Scholar] [RIS]
DOI: 10.1017/S0022381611000132

Bayes’ Theorem is increasingly used as a benchmark against which to judge the quality of citizens, but some of its implications are not well understood. A common claim is that Bayesians must agree more as they learn and that the failure of partisans to do the same is evidence of bias in their responses to new information. Formal inspection of Bayesian learning models shows that this is a misunderstanding. Learning need not create agreement among Bayesians. Disagreement among partisans is never clear evidence of bias. And although most partisans are not Bayesians, their reactions to new information are surprisingly consistent with the ideal of Bayesian rationality.

Published version
Appendix
Replication materials

[BibTeX] [EndNote] [Google Scholar] [RIS]
DOI: 10.1017/S0022381609090914

Although scholars have studied education’s effects on many different outcomes, little attention has been paid to its effects on adults’ economic views. This article examines those effects. It presents results based on longitudinal data which suggest that secondary education has a little-appreciated consequence: it makes Americans more opposed to redistribution. Placebo tests and other analyses confirm this finding. Further investigation suggests that these conservative effects of education operate partly by changing the way that self-interest shapes people’s ideas about redistribution.

Published version
Appendix
Replication materials

DOI: 10.1017/S0007123419000504

An enduring concern about democracies is that citizens conform too readily to the policy views of elites in their own parties, even to the point of ignoring other information about the policies in question. This article presents two experiments that suggest an important condition under which the concern may not hold. People are rarely exposed to even modest descriptions of policies, but when they are, their attitudes seem to be affected at least as much by those descriptions as by cues from party elites. The experiments also include measures of the extent to which people think about policy, and contrary to many accounts, they suggest that party cues do not inhibit such thinking. This is not cause for unbridled optimism about citizens’ ability to make good decisions, but it is reason to be more sanguine about their ability to use information about policy when they have it.

Published version
Preprint
Appendix
Replication materials

[BibTeX] [EndNote] [Google Scholar] [RIS]
DOI: 10.1017/S0003055411000165

By many measures, the public knows little about politics. But just how little people seem to know depends on the questions that are put to them. In particular, knowledge levels seem higher when people are asked closed- rather than open-ended questions. In turn, differences between estimated knowledge levels are sometimes attributed to fundamental differences between these types of questions. Building on this previous research, the present study uses a pre-registered experiment conducted with a representative national sample to shed new light on the relationship between question form and knowledge measurement. The authors find that inferences about political knowledge depend less on fundamental differences between open- and closed-ended questions than on two little-appreciated aspects of survey design: the number and difficulty of the response options that accompany closed-ended questions. These aspects of survey design have large effects. Scholars who use the same questions with different response options may reach substantively different conclusions about the public's levels of knowledge.

Article
Online-only appendix
Pre-registered pre-analysis plan
Replication materials

DOI: 10.1017/S0007123421000120

The study of voter competence has made significant contributions to our understanding of politics, but at this point there are diminishing returns to the endeavor. There is little reason, in theory or in practice, to expect voter competence to improve dramatically enough to make much of a difference, but there is reason to think that officials’ competence can vary enough to make large differences. To understand variations in government performance, therefore, we would do better to focus on the abilities and performance of officials, not ordinary citizens.

[BibTeX] [Endnote] [RIS]
DOI: 10.1080/08913810802316290

The study of voter competence has made significant contributions to our understanding of politics, but at this point there are diminishing returns to the endeavor. Voter competence is unlikely to improve dramatically enough to make much of a difference to our politics. By contrast, the competence of officials can and does vary substantially over short periods of time. To understand variations in government performance, therefore, we would do better to focus on the abilities and performance of officials, not ordinary citizens. We elaborate on this argument, emphasizing the “incompetence multiplier”: the way that the properties of hierarchies can amplify the incompetence of those in powerful positions. We illustrate our argument with an extended discussion of the U.S. response to the COVID-19 pandemic.

In some ways, this article is a sequel to our earlier Lethal Incompetence: Voters, Officials, and Systems.

DOI: 10.1515/for-2021-0010

Mediation analysis is the effort to understand the mechanisms through which some variables affect others. It is increasingly common in political science. But political scientists typically draw inferences about mediation without manipulating mediators, and their analyses are likely to be biased. Recognizing the problem, social scientists are gradually turning to methods that involve experimental manipulation of mediators. This is a step in the right direction, but experiments have little-appreciated limitations of their own. We describe these limitations and conclude that inference about mediation is fundamentally difficult—more difficult than inference about treatment effects, and best tackled by a research program that is specifically designed to speak to the challenges of mediation analysis.

[BibTeX] [Google Scholar]

The question of how causal effects are transmitted is fascinating and inevitably arises whenever experiments are presented. Social scientists cannot be faulted for taking a lively interest in “mediation,” the process by which causal influences are transmitted. However, social scientists frequently underestimate the difficulty of establishing causal pathways in a rigorous empirical manner. We argue that the statistical methods currently used to study mediation are flawed and that even sophisticated experimental designs cannot speak to questions of mediation without the aid of strong assumptions. The study of mediation is more demanding than most social scientists suppose and requires not one experimental study but rather an extensive program of experimental research.

[BibTeX] [Endnote] [Google Scholar][RIS]
DOI: 10.1177/0002716209351526

Scholars routinely test mediation claims by using some form of measurement-of-mediation analysis whereby outcomes are regressed on treatments and mediators to assess direct and indirect effects. Indeed, it is rare for an issue of any leading journal of social or personality psychology not to include such an analysis. Statisticians have for decades criticized this method on the grounds that it relies on implausible assumptions, but these criticisms have been largely ignored. After presenting examples and simulations that dramatize the weaknesses of the measurement-of-mediation approach, we suggest that scholars instead use an approach that is rooted in experimental design. We propose implicit-mediation analysis, which adds and subtracts features of the treatment in ways that implicate some mediators and not others. We illustrate the approach with examples from recently published articles, explain the differences between the approach and other experimental approaches to mediation, and formalize the assumptions and statistical procedures that allow researchers to learn from experiments that encourage changes in mediators.

Published version
Appendix
Replication materials

DOI: 10.1177/25152459211047227

Psychologists increasingly recommend experimental analysis of mediation. This is a step in the right direction because mediation analyses based on nonexperimental data are likely to be biased and because experiments, in principle, provide a sound basis for causal inference. But even experiments cannot overcome certain threats to inference that arise chiefly or exclusively in the context of mediation analysis—threats that have received little attention in psychology. We describe three of these threats and suggest ways to improve the exposition and design of mediation tests. Our conclusion is that inference about mediators is far more difficult than previous research suggests, and best tackled by an experimental research program that is specifically designed to address the challenges of mediation analysis.

Published version

Appendix

[BibTeX] [Endnote] [Google Scholar] [RIS]
DOI: 10.1037/a0018933

Commentary on the article by Eliot Smith, the new editor of Journal of Personality and Social Psychology: Attitudes and Social Cognition.

New standards for mediation analysis in Social Psychological and Personality Science. The editor, Allen McConnell, cites our article while establishing the new standards.

[BibTeX] [Google Scholar]

Partisanship seems to affect factual beliefs about politics. For example, Republicans are more likely than Democrats to say that the deficit rose during the Clinton administration; Democrats are more likely to say that inflation rose under Reagan. What remains unclear is whether such patterns reflect differing beliefs among partisans or instead reflect a desire to praise one party or criticize another. To shed light on this question, we present a model of survey response in the presence of partisan cheerleading and payments for correct and “don’t know” responses. We design two experiments based on the model’s implications. The experiments show that small payments for correct and “don’t know” answers sharply diminish the gap between Democrats and Republicans in responses to “partisan” factual questions. Our conclusion is that the apparent gulf in factual beliefs between members of different parties may be more illusory than real.

Published version
Online-only appendix
Replication materials

This article appears in Quarterly Journal of Political Science alongside a related article (“You Cannot Be Serious”) by Prior, Sood, and Khanna, who independently reached conclusions quite similar to ours.

In 2024, Manvir Singh offered a nice, brief discussion of this article in The New Yorker. Singh also discussed new work that yielded results like ours.

After these articles were published, a few objections were raised. Gabriel Lenz and I take up the objections in our Annual Review article, “Partisan Bias in Surveys”. See especially pages 332-33 of that article.

Some readers have objected to one particular question that we asked in our studies: a question about John McCain’s age. On these objections, see John McCain’s Age and the 2008 Presidential Election.

A previous version of this paper was released as NBER Working Paper 19080.

If citizens are to hold politicians accountable for their performance, they probably must have some sense of the relevant facts, such as whether the economy is growing. In surveys, Democrats and Republicans often claim to hold different beliefs about these facts, which raises normative concerns. However, it is not clear that their divergent survey responses reflect actual divergence of beliefs. In this review, we conclude that partisan divergence in survey responses is often not due to sincere, considered differences of belief that fall along party lines—but determining what it is due to is difficult. We review the evidence for possible explanations, especially insincere responding and congenial inference. Research in this area is still nascent, and much more will be required before we can speak with precision about the causes of partisan divergence in responses to factual questions.

Among other things, this review takes up objections to an earlier article, “Partisan Bias in Factual Beliefs about Politics.” See especially pages 332-33 of the review.

DOI: 10.1146/annurev-polisci-051117-050904

We now have a large and sprawling body of research on the effects of party cues. It is not very consistent or cumulative. Findings vary widely from one article to the next, and they sometimes contradict each other. This article sifts the evidence for five potential moderators of party-cue effects that have received much attention: political sophistication, need for cognition, issue salience, the amount of information in the information environment, and the distinctiveness of party reputations. It also considers the evidence on three large questions: whether party cues dominate policy information in people’s judgments, whether they are “shortcuts,” and how they affect our inferences about policies. The article closes by suggesting that limitations of research in this area are due partly to weak links between theory and empirical efforts and partly to problems of measurement error and statistical power.

DOI: 10.1093/oxfordhb/9780190860806.013.2

Transparency of research is a large concern in political science, and the practice of publishing links to datasets and other online resources is one of the main methods by which political scientists promote transparency. But the method cannot work if the links don’t, and very often, they don’t. We show that most of the URLs ever published in the American Political Science Review no longer work, and that the problem is severe for recent as well as for older articles. We conclude that “reference rot” limits the transparency and reproducibility of political science research. We also describe practices that scholars can adopt to combat the problem: when possible, they should archive data in trustworthy repositories, use links that incorporate persistent digital identifiers, and create archival versions of the webpages to which they link.

Published version
Online-only appendix
Replication materials

[BibTeX] [EndNote]
DOI: 10.1017/S1049096516002353