Example Reflection Papers

Published

October 21, 2024

Baseline Reflection (85 points)

Ansolabehere, Stephen, Jonathan Rodden, and James M. Snyder Jr. 2008. “The Strength of Issues: Using Multiple Measures to Gauge Preference Stability, Ideological Constraint, and Issue Voting.” American Political Science Review 102 (2): 215–32. https://doi.org/10.1017/s0003055408080210

Research Question

  • Ansolabehere, Rodden, and Snyder (2008) attempt to address what they see as an unsatisfying conclusion resulting from a vein of American survey research kicked off by Converse (1964): that the average American voter lacks preference on policy issues. Specifically, they investigate whether the temporal instability as well as seeming contradictions (the overall lack of constraint) associated with previous survey research really indicate that the average American voter has “incoherent and unstable preferences” or instead whether these results instead stem primarily from measurement error and, if so, how a survey researcher might lessen these measurement error effects.

Theoretical Framework

  • This paper contributes to a debate about whether people have consistent ideological attitudes at the individual level and how survey research can be used to assess this topic.
  • The paper identifies measurement error at the level of individual survey items as the core concept under inspection. The authors lay out a mathematical model of measurement error, assuming an underlying n-dimensional spatial model of ideology, with each dimension corresponding to a specific error and compute the bias and variance in the model, which informs their hypotheses.
  • Based on their model, Ansolabehere, Rodden and Snyder assert that previous studies have underestimated correlations between individual survey items due to measurement error. They then hypothesize that constructing measures by grouping issues together or using weighting will in turn reduce measurement error overall, and provide a better estimate of true correlations, and in turn, a more accurate picture of ideological constraint overall.

Data and Methods

  • The authors test their conclusions experimentally, by using sophisticated statistical analyses to quantify ideological preferences with “issue scales.”
  • They perform an analysis of several data sets sources from the NES. Specifically, they use “the 1956-1960, 1972-1976, 1990-1992, and 1992-1996 panel data sets.” They then compute the correlations between issue scales (scaled groupings of issues), panel waves (i.e., datasets taken at different points in time), and individual survey items to assess whether there is a significant difference in these correlations, i.e., a reduction of measurement error.
  • The primary outcomes of interest in the study are the computed correlations across issue scales and individual items, as well as the regression coefficients for models derived from the same quantities and whether these quantities are statistically significant as an indicator of constraint and/or measurement error.
  • The predictors used by the authors are party identification (as the baseline determinant for ideology) and experimentally the grouping or lack thereof of survey items. They also split up issues into “economic” or “moral.”.

Results

  • After computing these coefficients, the authors analyze their statistical significance, and in particular they analyze the statistical significance of the difference between the coefficients obtained from singular survey items vs. multi-item issue scales and thus obtain a measure as to how effective their interventions are in reducing measurement error. Results

  • Ansolabehere, Rodden and Snyder find that “voters have stable policy preferences” in contrast with the earlier work of Converse, and that the preferences of voters are in fact significantly affected by preferences on issues, also in refutation to previous work. They finally suggest that in order to obtain more meaningful results, researchers should focus on their surveys are measuring issues with multiple items and should devote significant effort to the overall quality of these surveys.

  • The most exhaustive item in the paper is Table 4, which gives the correlations for issues scales and individual survey items cross-tabulated with various demographic/informational factors. Table 3 gives the specific details for the issue scales vs individual survey items hypothesis, and
    Contributions

  • Ansolabehere, Rodden and Snyder find evidence that the drastic conclusions of earlier research; namely, that voters had little stability and consistency in their ideological preferences; can be refuted by explaining the variance in the data as measurement error instead. They also give suggestions for how to reduce measurement error in future studies.

Questions, comments and extensions

  • Overall, I find that the study presents a compelling narrative that gives some hope for the competency of the average American voter. The underlying mathematics used seem rigorous and so the paper is on a sound theoretical footing. However, the assumption that most of the variance in the data can be attributed directly to measurement error seems rather strong, which is a point taken up by Freeder et. Al (2018). As I have a background in probability and statistics, I would find a more thorough investigation of the underlying math quite interesting, to see if the mathematical assumptions made in the paper seem well-founded. Most importantly, though, this paper takes a much-needed positive viewpoint in the realm of ideology.

Reflection with Extension and Comparison (100 points)

Research Question

  • Jerit et al. (2006) ask what the importance is of the information environment on political knowledge. They address this question by distinguishing short-term, aggregate-level influences on knowledge from static, individual-level predictors. They argue that an individual’s political knowledge varies over time and is a result of both their static characteristics and their environment.

Theoretical Framework

  • The work of Jerit et al. (2006) contributes to a larger study of knowledge gaps which discusses the important role that stable individual characteristics including race, education, gender, and income play in identifying knowledge gaps. This field generally finds that changes in political knowledge disproportionately benefit the “informationally rich.”

  • This paper establishes the framework for later work that distinguishes between the type and temporal dimensions of political knowledge, and specifically how these differences are accrued through media preferences.

  • Similar to the other works in this area, Jerit et al. argue that people’s political knowledge will vary based on their static characteristics (mainly education) and the information environment in which they are exposed to such knowledge.

  • Jerit et al. (2006) present the following hypotheses (quoted from page 268): – 1a: Increases in the overall amount of media attention to an issue will increase the average amount of knowledge in the population – 1b: The gap in knowledge between individuals with low and high levels of education also will increase – 2a: All else held constant, increasing the amount of newspaper coverage will raise the average level of knowledge in the population, but it should primarily benefit those with high levels of education – 2b: An increase in television coverage will raise the average level of knowledge in the population, but it will not alter the relationship between education and knowledge

Data and Methods

  • Jerit et al. conduct an observational study of people’s political knowledge. They first measure people’s surveillance knowledge of 41 mainly health-related issues based on phone surveys. These surveys are paired with analysis of the frequency and manner of reporting on these issues across three news sources: AP, USA Today, and Broadcast TV News. They then analyze result through two studies:

    – Study 1 examines variation in the coverage of two separate issues using probit regression – Study 2 uses multilevel regression to examine both variation in coverage and of an individual’s education among multiple issues

  • In study 1, they modelled knowledge of a specific political issue based on the amount of coverage that the issue received. – The independent variable is how many stories were run about that topic – The dependent variable (the variable of interest) is people’s knowledge of that issue based upon their ability to correctly answer questions on the topic

  • In study 2, they modelled political knowledge of low and high coverage issues based on people’s education – The independent variable is people’s education – The dependent variable (variable of interest) is their political knowledge which is based off of the percent of relevant questions that they can correctly answer

  • The authors test and analyze this data using multilevel modelling in order to produce statistical estimates.

Results

  • The authors find that increasing media coverage of an issue increases political knowledge, but that increased coverage also exacerbates knowledge gaps
  • The key graphics are Table 1 and Figure 4

Contributions

  • While increasing media coverage of an issue is a valuable way of raising individual’s political knowledge of the issue, doing such tends to disproportionately benefit the informationally rich. This knowledge gap is made worse when the issue is reported on by newspapers and is slightly less of an issue when television is used.

Questions, comments and extensions

  • Jerit et al. provide a comprehensive survey of political knowledge and how it relates to media coverage and education level. They examine a large swath of public opinion surveys in order to field a far-reaching examination of the topic. Their use of many different sources and their statistical modelling and analysis was done well. One thing that could have been done better would be that they could use more recent topics.
    One of their data sources was public opinion surveys from 1990s. Presumably, by 2006, when the study was completed, people would have either forgotten about political events that took place in 1990, or, those events would have been so important that one would expect people to have a higher level of political knowledge on those than on other issues.

  • One lingering question that I have is how would the advent and popularization of Twitter and online news sources affect the outcome of this study?

“Revisiting the Gender Gap in Political Knowledge” Response

Jerit, Jennifer & Barabas, Jason. (2016). Revisiting the Gender Gap in Political Knowledge. Political Behavior. 39. 10.1007/s11109-016-9380-6.

Research Question

  • Jerit and Barabas (2016) ask whether male-female differences in political knowledge exist when proper survey instrumentation is used and whether these gaps can be ameliorated. They study how providing facts through information treatments can lessen or even get rid of the political knowledge gender gap. They seek to prove that the gender gap in political knowledge is “neither illusory nor immutable” (818). They argue that, unless a woman is genuinely disinterested in a topic, the knowledge gap is malleable.

Theoretical Framework

  • Previous research on the issue of the gender gap in political knowledge has largely focused on resource differentials and instrumentation. Some scholars in the field argue that the gender gap is not only real, but intractable, while others argue that it is nonexistent and simply a result of the survey measurements used (such things as question formatting and wording).

  • The core theory of this paper is that information treatments are adequate to minimize or even eradicate the differences between men and women in their political knowledge.

  • The specific hypothesis is that “the provision of information will help close the oft- noted knowledge gap between men and women” (821).

Data and Methods

  • The authors perform three experimental studies in order to test their hypothesis:

  • Study 1 experiments on two distinct groups: undergraduate political science students at a large public university and US MTurk workers (divided into 4 treatment and 1 control group). Each person was shown the same 3 news stories and the 4th was a randomized story out of 5 possibilities, each story headline and picture providing an information treatment. They were then tested on their knowledge of all 5 of the possible news stories having only received a treatment on one of them.

  • Study 2 replicates the findings of study 1 on a national scale and using different political stories.

  • Study 3 utilizes data from Barabas and Jerit’s 2010 study that examined the effect of naturally occurring information treatments on Medicare knowledge. They examine the knowledge gap between men and women before and after a 2007 government Medicare announcement in order to decipher how much the gap is affected by exposure to news coverage.

  • In all 3 studies, the independent variable is gender and the dependent variable is political knowledge

Results

  • Jerit and Barabas find that information, whether supplied in a controlled setting or in a “real-world” news treatment can reduce or even eliminate the gender disparity in political knowledge.

  • The key tables are figures are Figures 2, 4, and 5. Figures 2 and 4 show how the gender knowledge gap can be lessened or closed by information treatments. Figure 5 shows how much more valuable “real-world” information treatments are for men than for women.

Contributions

  • Despite longstanding evidence that the gender gap in political knowledge is either present and intractable or that it is nonexistent and the result of flawed measurements, Jerit and Barabas prove through their 2016 study that the gender disparity exists, but that it can be ameliorated through information treatments. Questions, comments and extensions

  • I thought this study was well done, thorough, and well explicated. Jerit and Barabas provide convincing data to show that the gender gap does exist, but that it takes a simple information treatment to reduce or eliminate it.

  • They do a good job of explaining their methods, including which news articles were shown and how those related to the questions being answered. Additionally, their data was made more convincing by the use of a control group. Had they only presented Study 1, I would have been unconvinced by the conclusion, as the people they were sampling were theoretically more educated than the average population; however, inclusion of Study 2 made the evidence much more credible.

  • My lingering question is how information treatments can be disseminated in a broad enough, “real-world” way (as in Study 3), that the gender gap in political knowledge could be eradicated.

Comparison

  • The two articles, both authored by Jerit, deal with the realm of political knowledge as a facet of one’s static identities. In the 2006 article, Jerit studies how an individual’s static characteristics contribute to a broad political knowledge gap, and how those on the more informed side – the “informationally rich” – are more benefitted by media coverage than others. In the 2016 article, Jerit studies a different political knowledge gap, one that is differentiated by sex (male versus female). The second article contributes to the findings of the first by positing how gender, which is a static characteristic, is deeply related to political knowledge. Unlike the 2006 study, which is mainly observational, the 2016 study is an important contribution to the field of political knowledge research because it offers tested and statistically significant evidence that the knowledge bap between men and women is malleable. This 2016 paper brings to the forefront the question of whether the knowledge gap presented by Jerit (2006) between the informationally rich and the non-informationally rich is as malleable as the gender gap, and under what circumstances the gap can be lessened or even closed. It seems likely that if simple information treatments can render women as knowledgeable on political topics as men, then similar information treatments should be able to raise the knowledge of the informationally poor. The question is, what information treatments would be needed to go about narrowing this gap and how can those treatments be spread to the general public. This is an important question and contributes to a larger and more important concern about how to increase the number of people who vote regularly in US democratic elections and, more broadly, how to improve citizen competency.

Using what you’ve learned about public opinion in the course so far, make the case for or against citizens’ competence in a democracy.

Whatever side you take in this debate, present the best evidence for your argument, consider the strongest objections to that argument, and reassure us that your claims stand inspite of these arguments.

Along the way, you may want to offer some discussion of what you mean by competence, and the various roles that citizens and their opinions might play in a democracy. You should conclude by discussing the implications of your argument for democratic theory, politics, and policy.

You may structure your paper how you please, but something like:

  • Introduction
  • Argument
  • Critique
  • Response
  • Conclusion

will make it easier for your reader (and grader) to follow along.

You must cite readings from the course, and you should offer some detailed discussion of the actual arguments these authors make, the larger theoretical literature they speak to, and the specific evidence they use to make their case. Don’t just say “So and so says citizens lack coherent belief systems” and move on. Describe succinctly what they mean by beleif systems, why such a concept matters for your larger argument, and the main evidence the author puts forward in support of their claim. Similarly, simply listing a bunch of studies and tagging each as evidence for or against citizen competence, will be a less effective strategy presenting works in dialogue with each other. You may bring in readings from outside the course but must cite them, and more importantly, use them correctly.

You have 10-pages double-spaced (not including references) to accomplish this task. You will need to make tradeoffs between the breadth and depth of your discussion. You may include tables and figures (these will not count toward your final page count) from referenced articles, but should only do so if it is vital to making your argument (I am fairly certain it is not).

I will gladly answer clarifying questions about specific readings from the course. We will not answer questions that begin or end with “is this right?” That said, if you walk us through your interpretation of a passage or result and explain your reasoning for reaching a conclusion, we will happily discuss that process with you.

Grading Rubric

Here is a link to the grading rubric we will use to grade your term papers