RESEARCH METHODS IN POLITICAL SCIENCE


COURSE HOMEPAGE     STUDENT RESOURCES



Please note: At your instructor's discretion, there may be minor alterations to the reading assignments listed below. One of the major advantages to providing you with an on-line readings archive is that timely articles can be added or substituted when appropriate. Opening documents downloaded from this website will require that your computer have Acrobat Reader . You will also need the class-specific password to open individual files.

UNIT 1 ASSIGNMENT SCHEDULE

Links to helpful resources:

 

Week 1

Topic 1 (Monday 8/19)—Why are you taking this class, and what can you expect to learn?

  • After class, please carefully read through the documents above so you are familiar with several resources that you will want to consult multiple times during the term.

Topic 2 (Wednesday 8/21, Friday 8/23)—What is "empirical" research, and what kinds of questions do political scientists seek to answer?

Ahead of class on Wednesday:

  • Read chapter 1 from your textbook. Looking at the list of topics the author provides at the end of chapter, think about what kinds of questions attract the attention of social scientists and how they structure studies to answer them.

  • Take a look at what studies have been published in American Journal of Political Science : https://onlinelibrary.wiley.com/toc/15405907/current. Read through a handful of abstracts (follow the abstract link to see a quick summary; do no read full articles). If you are an INR major, look at recent studies that examine other countries or internal issues. PSC majors, take a look at anything of interest. 

  • Take a look at the draft set of focus questions I've placed in the PPT and Assignments file. There will be some changes to this draft document as things unfold during the term, but this will give you an idea of how you should be reading for the class.

For Friday:

  • Read quickly chapter 3 in your textbook. The main reason for asking you go through this reading during the first week in the class is that doing so will help you to begin to absorb vocabulary and concepts that will come up throughout the course. We will cover all of the concepts identified in the chapter multiple times, and get to some of them this coming week. You will be asked to read the chapter again.

Read the chapter sections on "dependent" and "independent" variables very closely. Also carefully review the sections on how researchers frequently look for "correlations" between independent and dependent variables to test their "expectations" (that is, their "hypotheses") to "explain" (that is, to test their hypotheses to see if the correlations confirm their theory) about the relationship between independent and dependent variables. For the section on "variable measurement," you can skim the sections about and definitions of "nominal," "binary," (also called "dummy), "ordinal," "interval," and "ratio" variables. We'll talk a lot about these throughout the term, but these are concepts we'll begin to talk about in the coming weeks.

  • Read an article by Madison Deane '2024 (Just read up to the findings section; we'll go over the findings and conclusions in class). Bring a copy of the article to class so you can reference it. You are used to reading course assignments to walk away with their main conclusions and evidence. Here, I want you to focus on the article's methods and how the author goes through each of the steps outlined in Chapter 1 of your textbook. The reading from chapter 3 will give you some of the vocabulary you will need to navigate this research article. You aren't expected to understand everything in this article--it is written for an academic journal whose readers mostly have advanced research methods training, but do your best to answer these questions:

    • What are main research questions, and why does the author think they are important questions?

    • What is her theory about the relationship about a person's attitude about fate and support for social assistance?

    • How do she measure different types of beliefs people may have about why bad and good social outcomes happen? How does she measure fate?

    • What specific hypotheses do they posit to test these theories and what methods (i.e., interviews? content analysis? survey data analysis?)?

Week 2
Topic 3 (8/26, 8/28, 8/30)—Why do we want to study politics "scientifically"? What do we give up when we focus most of our effort on being empirical rather than political or normative in our approach to understanding politics?

For Monday

  • If you were waiting for your book to arrive last week and didn't complete the assignment, read though your textbook's chapter three (a copy of that chapter is in a folder in the PPTs folder). Why are you reading this chapter so important? I want you to be familiarizing yourself with the basic concepts--theory, hypotheses, and different types of variables--used in social science research. If you don't understand these terms, it will be hard to grasp what the sample articles we are reading are trying to show.

  • We'll also talk a little more about the research article you were asked to read by Madisone Deane.

  • John R. Bond, "The Scientification of the Study of Politics" (Journal of Politics, 2007, 11p). This is a difficult reading (it was written for a leading political science journal). Just skim the specifics on the books and articles he mentions, so that you have a general understanding of the context he's writing about. Concentrate on when and why political science moved towards mostly being an empirical social science and some of the debates over that shift. What makes the present-day discipline of political science "scientific," and what's the alternative? How does Bond know and demonstrate when political science (and sociology and international relations) became more scientific? You don't need to know specifics, but be familiar with the main arguments.

For Wednesday

  • Be warned that this outtake from the John Oliver show is crass in places, but it goes over lots of concepts that you will find helpful to getting the most you can out of your other readings this week. Ignore anything not related to how science is used and reported on in the US
    https://www.youtube.com/watch?v=0Rnq1NpHdmw. I am assigning this content because Oliver is funny enough to help you remember some key ideas about how some social scientists produce studies that aren't as reliable as you might thing. Here are some questions to guide your viewing:
    What is p-hacking? Why is there so little replication in the sciences, let alone the social sciences? What are some of the factors that lead to separate studies finding opposing results? Why do findings so often get distorted when when they are publicly disseminated? Does all of this mean that we just can't really build knowledge through the scientific approach?

For Friday

  • Review chapter three in your textbook, reading it more carefully this time. Make sure you understand the different types of variables Forestiere defines and discusses. One key type of variable not discussed in the chapter are control variables. This coming week, you will be reading about two types of control variables.

  • Catch up if you haven't completed the other reading assignments for this week. Remember to use the study guide to help you stay focused on what you are supposed to be getting out of the readings. You do *not* need to read much of the assigned reading very closely if your take the time to see in advance what I would like you to review carefully.


Weeks 3 and 4


Topic 4
—How to you turn an interesting question into a doable research project?: Operationalizing a study's theories, hypotheses and causal arguments

Week 3: Sept. 2, 4, and 6


For Monday (9/2):

  • Read Chp. 4 in Carolyn Forestiere's Beginning Research in Political Science (Your textbook). Read most of this chapter very closely. This material covers most of the key research-design concepts and terms used by social scientists, regardless of whether they are using quantitative or qualitative methods. The chapter illustrates these concepts with a chapter-length example. Be familiar with each element of that example. You do not need to read the "Examples from Political Science" section here. We are looking at article-length examples in the other reading assignments and class.

  • Read this Wall Street Journal article, focusing on the methodology (mostly experiments) social scientists are using to examine political polarization and its consequences. Specifically, how are the social scientists operationalizing their variables to test the idea that Americans are using political information in their daily lives in tribal ways, that is to favor copartisans and punish folks who identify with the other party? Why might experiments be a better way to examine polarization than just asking people what they think about different types of partisanship (there is a chart that looks at this, too, in the article)?


For Wednesday (9/4):

This is a tough reading that was not written for an undergraduate audience. Do not spend great energy and time reading the results or information about the study's statistics closely.

Instead, look at the main sections of the article to get a sense about how published social science studies typically are organized. See if you can identify the study's main research questions, the study's hypotheses, and its general findings.

Focus on the methods. What kind of survey did the authors use? How did they measure different elements of tribalism with "games," and what evidence do they provide to suggest that perhaps tribalism can be reduced (i.e., how do they test hypotheses related to "contact theory?)?

Review the big-picture takeaways? How did tribalism in the US as of 2019 compare to what other scholars had reported in earlier years? How did divisions as of 2019 compare to what Dr. Whitt has found in his studies of other international areas?

  • Here are the experiments that approximately 1,200 Americans answered in this study (the description of them in the article is very brief because this journal has rigid guidelines on how long an article can be). Read through them quickly. We'll examine them in class, too.


For Friday (9/6) and continuing into Monday (9/9)

  • Read closely parts of Chp. 5 in Carolyn Forestiere's Beginning Research in Political Science (Your textbook). For now, read the introduction of the chapter, skipping all of the sections on sampling (pp. 91-99 will be assigned next week). Then, read the section "Revisiting the different types of research design" through the end of the chapter.

    • What is the central research question? What is the basic set-up of the two types of research they use? Hint: What type of experiment do they use and what type of elite interviewing? What can each type tell you that the other can't by itself?

    • What does the theory in article tell them about the relationship between persons' socioeconomic status and their likelihood of being victimized by corrupt officials?

    • What are the authors' main concepts/variables and how are they measured (aka "operationalized")? What are the dependent and independent variables used to measure their main concepts?

    • What are the main hypotheses and how (in very general terms) are they tested? What is the null hypothesis?

    • How do the authors "control" for the effect of certain variables (age and gender) to isolate the effect of the variables they care about? [Important: See note below where I compare how controls used in an experiment like the one in Mexico City with "statistical controls" used when researchers are working with a large number of observations (which is the case in the first article you read by Madison Deane and the other articles assigned as examples).]

    • What are the authors' main findings? In what way is this research "quantitative" as well as qualitative?  

    • Are their any obvious limitations to their research and findings? For example, the article's title suggests that this study advances our understanding of how "inequality" influences "corruption" in "Latin America," but how reasonable is this claim considering the actual experiment they use and unique qualities to their case study (i.e., some road intersections in Mexico City)?


Week 4
: Sept. 11, and 13

For Wednesday (Sept 11):

  • Read (please print it out, too, and bring it to class): Mark Setzler, "Did Brazilians Vote for Jair Bolsonaro Because They Share his Most Controversial Views?"  Use the appropriate link to download a pdf-version of the full article. Although this article uses pretty advanced statistical methods, I assign it because it is pretty short (that's what a "research note" usually is), and it was written to be easily accessible to an audience that includes undergraduate social science majors. Brazilian Political Science Review is published by the leading academic organization for political scientists in Brazil and publishes all work in English to make it accessible to the widest audience possible. As a reminder, the main reason you are reading some works written by your instructor is because the assigned studies were written with HPU students in mind and mostly use only the same statistical techniques we teach in PSC 2019 and the department's 4000-level courses. The other reason you are being asked to read this article is so that you can focus on how scholars create variables from survey questions to measure things like, how sexist a voter is or how much they support democracy.

    As you read, consider the questions listed for the last reading as well as the following:

    • What variables are used to test the main two theories for why Brazilians voted for Bolsonaro (controversial issues vs. standard issues)? What hypothesis is the author most interested in (hint: it is noted in the article title)? What theory represents the null hypothesis?

      Notice that the author doesn't use control variables because he basically groups them into the second hypothesis, which is that the "standard" issues explain Bolsonaro's election. If he had only been testing the hypothesis that voters' with illiberal beliefs were more likely to vote for Bolsonaro, variables like gender, age, education, income, ideology, etc., would have still been used in the article's regression models as controls. Why? Because each of them could be correlated to the main independent variables in the study and the probability of voting for Bolsonaro.

    • What specific questions were used as indicators to test the main hypotheses? What question was used for the dependent variable? What questions' answers were used as independent variables?

    • Looking at the different questions used in the study, which variables were originally dichotomous (also called a "dummy" category)? Which were interval? Which were ordinal? Which were nominal (also called categorical)? How were these variables changed prior to being analyzed?

    • The study considers also whether gender and race were intervening variables in determining who voted for Bolsonaro. Did the influence of variables that shaped a person's likelihood for Bolsonaro differ by a person's gender or race?


For Friday (Sept 13):

  • Reread the section on "Comparative Studies" (pp. 107-111) in your textbook's Chp. 5. Your reading should include the two "Examples from Political Science" for "most similar and "most different" case-study research.

    • What is the central research question? What is the basic set-up of the two types of research he uses (specifically, why does the author not only look at religiosity in Latin American but patterns in four specific countries, too)?

    • What does the theory in article tell us about the potential relationship between persons' religiosity and their likelihood of being thinking men make better political leaders?

    • What are the author's main concepts/variables, and how are they measured (aka "operationalized)"? 

    • What are the main hypotheses and how are they tested? What is the null hypothesis?

    • What are the control variables in this study? The main argument being tested is that voters share the controversial views of the politicians they support. This means that the "standard" issues variables are acting as controls in this case. 

      Important note, you need to make sure you understand what a control variable is. The article you read on bribing in Mexico included experiments that included only with men in their 30s. This "most similar case study" approach allowed the researchers to isolate the effect of a person's class, versus their age or gender, on their likelihood of being asked for a bribe. In experiments, researchers often control for other potential influences on an outcome by making sure that all of their subjects are exactly the same on some factors so that they know that these factors are NOT causing different outcomes (the controls in the Mexico study were age and gender, which didn't vary so the researchers could "isolate" a person's socioeconomic status on whether they are asked for a bribe).

      In most of the studies you have been asked to read so far in the class, the authors are analyzing a large number of observations and using statistical controls. Statistical analysis programs isolate the influence of an independent variable by looking at across different clusters of a control variable. For example, we could use a well-known, recurring survey of Mexican public opinion--the AmericasBarometer--to reexamine what the authors studied in Mexico city. If we were statistically analyzing the effect of income on being asked by the police for a bribe, we would use questions in the 1500-person survey dataset that asks each respondent about these two things. We could control for age by instructing the stats program to cluster respondents by responses to an item asking respondent's their age. If income--rather than age--explains how the police treat Mexicans, we would expect to find that rich Mexicans are less likely to be asked for a bribe than poor Mexicans at every age. On the other hand, we might find that older people tend to have more money, but when you compare people grouped by age, older Mexicans get asked less for bribes regardless of SES while younger Mexicans get asked for bribes more ofent regardless of status. If this were the case, we would say that being younger is a major cause being asked for a bribe while SES is not. Statistical controls are somewhat more complicated than what I have just laid out, but this is the general idea of what is going on when we use statistical programs to isolate the effect of a variable on an outcome when multiple factors could be in play.

      So, when does a study need a control? Whenever you think that there may be something else that could be influencing the independent and dependent variable. In medical studies, for example, people might get better if they think they are taking a medicine to improve their health. To make sure that it is the new drug that is resulting in better health outcomes, scientists give some of their sample a placebo (a fake drug that does nothing); by doing so, .
         

    • What are the author's main findings? Are their any obvious limitations to his research and findings? What other methods (including qualitative social science) could a researcher use to study the relationship between religiosity and attitudes about women as political leaders?. 


Week 5

For Monday (Sept 16): Workshop day

  • Review textbook chapters 4 and 5 (skipping the parts on sampling, which we will discuss next week) if you have not already read both closely and taken notes on all concepts listed in the study guide.

  • In class, you will work in teams to design a research study or two. If you had ample resources at your disposal and could collect original data, what kind of "mixed methods" (in this case, quantitative and qualitative) research would you use to study a political or economics research question that seems interesting to you?  


Topic 5 —Surveys and samples--studying a relatively small number of people to make accurate inferences about much larger "universes”

For Wednesday (9/18):

  • Read the rest of Chapter 5 in Carolyn Forestiere's Beginning Research in Political Science (i.e., read only the several-page section on Sampling and Generalization (including the sub-sections).

  • Read the very short Chapter 6 in Carolyn Forestiere's book. Focus on the what makes a study measure (i.e., a variable, which may also be called an "indicator") reliable and valid


For Friday (9/20). It may seem like a lot of readings are listed here, but these collectively add up to less reading than a textbook chapter. Each piece is the equivalent of a short textbook chapter section. Use the focus questions to guide your reading.

    • I assign this short document so you can see best practices in survey composition. The ATP is a meticulously created stratified random sample panel of respondents who are compensated to stay in the survey over time. 

    • The document explains how this sample was designed to ensure that every American had the same probability of being asked to participate. What type of random sample is the APT (hint: the full panel is a stratified random sample with some oversampling) What's that mean)? How were individuals contacted to participate? How big is the sample? What was the margin of error for the sample (at a 95% confidence interval)

    • What parameters were used to ensure that this sample closely matches the characteristics of the nation as a whole?

    • What did Pew do to incentivize selected individuals to participate? What percentage of contacted people take the survey?

    • Which subgroups were over-sampled? Specifically, Pew invited extra individuals from several subgroups whose members are less likely than other individuals to participate in surveys or because they wanted researchers to have access to a subgroup samples that more accurately reflected these subgroups overall. 

    • How do Pew analyses of their surveys deal with the fact that even the most carefully created random samples typically do not mirror their target population exactly because different types of people are more or less likely to accept invitations to participate. To deal with this issue, researchers typically apply post-hoc (i.e., after-the fact) weights that use parameters from the Census or other massive, high-quality surveys. Specifically, researchers' data analyses typically under-weight or over-weight the responses of different types of individuals based on whether their were too few or too many respondents of each type in the sample relative to their share of the larger population. In Pew's case, they used population parameters to determine who would participate in their survey and--because they purposefully oversampled some groups--they also use post-hoc weights.


Week 6

Monday, September 23: In-class Exam 1.   The exam will contain 10-15 multiple choice and several short-essay items. If you have previously approved arrangements with OARS for extended or distraction-free testing time, you must contact them a week in advance to ensure that you can take the test is the OARS testing facility. In booking a room, your exam must start at the same time as our class begins; any exceptions to this must be pre-approved by me.   

Topic 6 (Wednesday and Friday, Sept. 25, 27, after the test): Is political polling getting better over time? Can we still trust surveys to determine what Americans think about contentious issues?

  • Stef W. Kight and Margaret Talev: "Study: What Americans really think" (Axios 2022, 3pp.)  This article uses the phrase "self silencing" to refer to what social scientists refer to as "social desirability bias." What is a list experiment? How does it tackle the problem of this type of bias? While we know that people don't always admit to their prejudice against other group, what does this article (and data in class PPTs) say about how widespread social desirability bias is?

The next several articles discuss how political polling has improved over the last several elections. It looks like a lot of reading but together, it adds up to less than 15 pages.

  • Geoffrey Skelley, Why Was The National Polling Environment So Off In 2020? (FiveThirtyEight, 2021, 3-4pp). How far off are polls from estimating actual turnout and vote choice in the typical election? What is non-response bias? This article doesn't talk about it, but what are some of the factors that might cause the supporters of Donald Trump to be more or less likely to take a survey than Democrats, independents, and even some Republicans who don't support him?

  • Andrew Fischer, "Polls Were Great in 2022. Can They Repeat Their Success" (NY Times, 2pp). What are polls doing to get better? How has adjusting polls to make sure that they have an proportionally accurate share whites and non-white individuals without a college education and respondents who voted for Trump in the previous election improved polls' predictions. 

  • You might be interested in reading more (i.e., this is optional, but only because it is so detailed) about how accurate political polling typically is and whether there is a political bias to polling (when large numbers of surveys are combined, there typically isn't, especially if you weight for "house effects," which fivethirtyeight.com does): https://fivethirtyeight.com/features/2022-election-polling-accuracy/


We finished up Unit 1 when we took its exam. I posted materials for the rest of the week on this schedule so you would know what we would be doing right after the exam. As we continue with Unit 2, head back to the course homepage and open up the Unit 2 schedule.