Social Science Methods Syllabus

Course Code:
Education Cycle: Second cycle
Main Field(s) of Study and In-Depth Level: Political Science A1N, Development
Studies A1N
Grading System: Fail (U), Pass (G), Pass with distinction (VG).
Established: 2007-01-24
Established by: Samhllsvetenskapliga Iakultetsnmnden
Revised: 2011-04-15 Revised by: Allan DaIoe, Katrin Uba
Syllabus Valid From: week 37, 2011
Entry Requirements: Political science advanced level, Development Studies advanced
level or equivalent education.

Content
Is democracy a Iorce Ior peace in the world? Do austerity policies increase the probability oI
civil unrest? Why are some countries poor and others rich? What accounts Ior the diIIerent
political systems present in the world, and what eIIects do these have on wealth, happiness, civil
violence, and international stability? It is important to try to answer these and other questions.
Understanding causal processes in social systems, however, is diIIicult: systems are complex,
people are heterogeneous and adaptive, and it is generally inIeasible to carry out experiments.
This course will introduce you to the tools oI modern social science to help you evaluate and
develop research relevant to these important questions.

The course discusses the diIIerent stages oI the research process and the critical choices that
a researcher is conIronted with. What does a good research question look like? What criteria
should guide theory Iormulation? What are the strengths and weaknesses oI diIIerent research
designs? How should empirical data be interpreted? The course discusses, among other
things, the principles oI inIerence and eIIective research, the criteria oI good theory, the role
oI the mathematical Iormalization oI theory, the purpose oI and examples oI good research
design, qualitative methods such as case-studies, small-n comparison, and causal-process
observation, statistical methods Ior descriptive and causal inIerence, multivariate linear and
logistic regression, graphical techniques Ior presentation oI data, matching, natural experiments,
instrumental variables, and multiple-comparisons and publication bias.



Learning Outcomes
AIter completing the course, students are expected to:
Be aware oI the basic principles underlying descriptive and causal inIerence.
Understand and be able to assess the criteria that make up a good theory.
Be able to articulate new theory and revise existing theories in an eIIective manner.
Be Iamiliar with the language, purpose, and tradeoIIs oI Iormal theory.
Understand the strengths oI diIIerent kinds oI qualitative research, including case-studies,
small-N comparisons, qualitative comparative analysis, process tracing, and causal-
process observation.
Be able to apply various kinds oI statistical inIerence, with awareness oI their
assumptions.
Be able to create eIIective graphical presentations oI data and judge those Iound in the
media, and in policy and academic publications.
Be aware oI the diIIerent strategies Ior identiIying and estimating causal eIIects, including
multivariate regression, matching, exploiting use oI natural experiments, and causal-
process observation.
Have thought extensively about the application oI these methods in diIIerent research
domains, and developed some design ideas Ior their own research.


nstruction
The course consists oI a number oI week-long segments involving lectures, mandatory seminars,
and weekly homework. The course literature consists oI books, articles, and practical examples.
Some oI the homework will involve using statistical soItware (Stata); copies oI Stata can be
purchased Ior a reasonable price through expedition, or students may use the Stata-enabled
computers in the computer lab.

Readings
The two required books are:
Kellstedt, Paul and Guy Whitten. 2008. The Fundamentals of Political Science Research.
Cambridge University Press.
George, Alexander L. and Andrew Bennett. 2004. Case Studies and Theorv Development in the
Social Sciences. MIT Press.

The rest oI the readings consists oI articles and chapters in books made available via
Studentportalen, directly downloadable, or available at the library.

Assessment
The grading oI the course will be based on completion oI weekly homework, active seminar
participation, two examinations, and a research-design project. Participation in seminars is
obligatory. II you miss one Ior a justiIied reason, you will be required to answer some extra
seminar questions. Note, even iI you miss the deadline you still need to hand in your homework.
Homework will be graded only as pass (G) or Iail (U).

The examinations will cover the material covered in lectures, homework, and readings.
Final grades are awarded on a scale comprising the grades VG (pass with distinction), G (pass),
and U (Iail). The exams and RD assignment will each be worth 50 points. To reach the grade
pass (G), students must have handed in all homework and extra assignments Irom missed
seminars/homework, and passed the exams and RDA. A passing grade on an exam is a score
25/50. A G on the RDA is worth 30 points, a VG is worth 50 points. A pass with distinction
(VG) will be awarded to students who have more than X points on the course, have handed in all
homework and extra assignments Irom missed seminars/homework, and have not Iailed the RD
assignment or exams. The VG threshold (X) will be calibrated at the end oI the course so that a
VG is comparable in achievement to a VG Irom previous years in this methods course; X will be
at most 135.

Guidance, Help, and Feedback
The large size oI this course will make it diIIerent in some ways Irom the typical smaller course.
First, it will be important Ior you to get to know others in the class. You will meet each other
through your seminar group, but you should also make a point oI introducing yourselI and
exchanging emails with other classmates. Then, iI you have questions about logistical issues,
you can Iirst ask them. Also, it will be helpIul to talk and think through the content oI this course
with others. It is encouraged to work with others in understanding the material, studying Ior
exams, and on homework questions (so long as your homework is ultimately done by yourselI).

Your next point oI contact about understanding material should be your seminar leader, who
you will see once a week. II you have substantive questions, we recommend that you write them
down, email them to your seminar leader in advance oI your seminar, and then, time permitting,
your seminar leader may be able to assist you with this question during the seminar.

Logistic questions about studentportalen should be directed to the appropriate course
administrator in your department.

Finally, any remaining speciIic logistic or substantive questions that require an answer
can be directed to the conveners. You may come speak to us aIter our lectures, or email us
your question. II you need a longer meeting, email us and we will arrange a time. Because
there are two conveners, iI your birth day is an even number, please email Allan DaIoe
(allan.daIoepcr.uu.se); iI your birth day is an odd number, please email Katrin Uba
(Katrin.Ubastatsvet.uu.se).

We do very much want to receive Ieedback about how the course is going, and questions and
thoughts you have about course material. Please Ieel Iree to send us emails with your thoughts,
input, and questions. Time prevents us Irom habitually responding to these, but we will take your
Ieedback into account, and, iI appropriate, we will respond to the questions and issues raised in
these emails during our subsequent lectures.

Seminars
There will be a weekly seminar where you will have the opportunity to discuss topics
related to the lectures, readings, and homework in a smaller setting.
Your weekly homework is due at noon on the Saturday beIore your seminar. Failure
to hand in homework on time will be noted, and may require you to do an extra major
assignment.
Attending your weekly seminar is mandatory. II you are unable to attend, you must
inIorm your seminar leader in advance oI missing the class, and explain your absence.
UnjustiIied absences will be taken into account when determining the Iinal grade.
You may be asked to do an extra major assignment to make up Ior excessive absences
(justiIied or not).

Structure of Lectures
Lectures are not mandatory, though material presented in them may be on the exam. II
you choose not to attend, it is your responsibility to get notes Irom your classmates. For
some lectures, lecture notes and slides will be posted online.
Please be on time so that you can get settled by the time the lecture begins. The lectures
will start promptly at their designated time.
Electronic Devices: We would like everyone to have the opportunity to use laptops
or other devices Ior the purposes oI taking notes, looking up relevant material during
lectures, and in other ways conducive to your involvement in the lecture. However,
such devices also generate a temptation to distraction---checking email, Iacebook, etc...-
--which detracts Irom class engagement and energy. Our solution to this is to share
our understanding about the ethics oI classroom attendance: we invite you to use such
devices, but not Ior activities that don`t contribute to your or others` learning experience
in the class. We understand, then, that iI we see you using a laptop or other device, that
you are making a promise to us that you are using it Ior learning purposes or otherwise
urgent matters. II you are not able to make such a commitment, do not use these devices
in class. Since lectures are not mandatory, iI you choose to be physically present we
expect you to be mentally present as well. Also: mute or turn off all devices capable oI
making disruptive noise.
Participation: You are encouraged to raise your hand at any point during the lecture iI
you have a question, and to raise your hand to respond to questions posed by the lecturer.
Asking questions provides us with Ieedback, and your question or uncertainty is almost
certainly shared by others. Additionally, you may email questions or thoughts you have
to the conveners; we may then respond to these in subsequent lectures and we will avoid
mentioning the names oI the authors oI these questions.

Homework
There will be weekly homework exercises. These should be uploaded no later than Saturday
at noon, to studentportalen. They should be uploaded as pdI Iiles (as an option, you may in
addition upload it as a .doc Iormat), and typed as much as possible on a computer (Times New
Roman or similar Iont; 12 points; spacing at 1.5 lines). II you want to add hand-drawn Iigures
or otherwise do something by hand, consider scanning or taking a digital photo oI your hand-
drawn components, and then digitally insert it into your document. II you absolutely must do the
homework by hand, then do so, and upload a scanned pdI oI the Iinal copy.

This homework is intended to take you some time to do, and some oI the questions are intended
to be challenging, as it is your opportunity to wrestle with the material in the course in a ``saIe"
setting. You should work on the homework in the Iollowing manner. (1) First, work on them,
struggle, agonize, by yourselI. Do your very best to answer all the questions on your own. This
is when most oI your learning will take place. (2) Then, aIter you have put in a good eIIort Ior
each question, you should consult with your classmates to compare your answers. Discuss the
reasoning behind each attempt. Help each other. Cooperate. (3) Finally, write up your Iinal
homework by yourselI, representing your own thoughts and understanding, written in your own
words. Do NOT hand in material that is identical to your classmates (unless it is a math question
Ior which there is only one right answer); that will be regarded as plagiarism. Obviously aIter
cooperating with your classmates you will converge on similar answers; however, you should
explain these in your own words, using your own examples, and ideally, your answers will
diverge somewhat reIlecting your own diIIerent understanding.

So long as substantial independent eIIort is shown, the homework will be awarded a ``G". An
answer key will be provided Ior you to assess your answers, though you should also discuss the
questions with your classmates and seminar leader to make sure you understood the material.

Examinations
There will be two in-class examinations, one Ior each 'halI oI the course. The Iirst exam will
cover the material---lecture material, homework, and readings---occurring in the Iirst part oI the
course; the second exam will cover the entire course, but will be heavily weighted towards the
material Irom the second halI oI the course.

The exams are meant to test your understanding oI the concepts and issues in social science
methodology. We will avoid questions that involve rote memorization, so you need not worry
too much about memorizing deIinitions or equations. However, you should strive to eIIectively
employ relevant terms and concepts, and as such it is important that you are Iamiliar with the
key terms and concepts in the course. The Ilip-side oI this is that it should not be possible to do
well on the exams with just rote-memorization; it is much more important that you understand
the basic conceptual issues involved, then that you can reproduce speciIics. The exams will
involve questions about material covered in the lectures, homework, and readings. Mathematical
competence is not required to get a G on the exams or in the course, though it will help to excel.

Research Design Assignment
There is one major Research Design assignment in this course. See the associated handout Ior
details about this. For those interested in a potentially more challenging and more academically
rewarding assignment, you have the option to do a Replication Assignment; see associated
handout. You will discuss this assignment during some seminars, and you should discuss your
progress on this assignment with your classmates, giving each other Ieedback and suggestions.
Working on this assignment should (1) help you think about the utility oI diIIerent methods to
your Iuture research (MA thesis), and (2) help you start thinking about and getting Ieedback on
ideas Ior your MA thesis.

Reading-list for the Methods Course for the Master Programme in
Politics and International Studies, Uppsala University
Convenors: Allan Dafoe (Allan.Dafoe@pcr.uu.se) and Katrin Uba (Katrin.Uba@statsvet.uu.se)
Required books:
Kellstedt, Paul and Guy Whitten. 2008. The Fundamentals of Political Science Research.
Cambridge University Press.
George, Alexander L. and Andrew Bennett. 2004. Case Studies and Theory Development in the
Social Sciences. MIT Press.
All required reading will be uploaded to the Studentportalen or will be available via Up-
Unet i.e. search via library or scholar.google.com.
Introduction: Overview
Required Readings:
1) Kellstedt and Whitten: Chapter 1
2) Excerpts from Sagan, Carl. 1997. The Demon-Haunted World. Random House, Inc.
3) Gerring, John. 2012. Social Science Methodology: A Unified Framework: Chapter 1
(Discovery and Appraisal) and Chapter 2 (Toward a Research Question):
Recommended Readings:
1) Platt, J. R. 1964. Strong Inference. Science.
2) C Booth, Wayne, Gregory G Colomb, and Joseph M Williams. 2008. The Craft of
Research: Prologue, Ch1, Ch3, Ch4:
3) Jervis, Robert. 2002. Politics, Political Science, and Specialization. PS: Political
Science and Politics.
4) King, G. 2006. Publication, publication. PS: Political Science and Politics.
5) For example of output from a replication assignment, see:
Dafoe, A. 2011. Statistical Critiques of the Democratic Peace: Caveat Emptor.
American Journal of Political Science 55(2): 247262.
6) For a very short intro to Bayesian thinking:
Paulos, John Allen. 2011. The Mathematics of Changing Your Mind. The New York
Times.
Introduction: Empirics
Required Readings:
1) Dafoe, Allan & Devin Caughey. Honor and War: Using Southern Presidents to Identify
Reputational Effects in International Conflict.
2) Kellstedt and Whitten: Ch3
3) Gerring 2012: Ch 4 Analysis
4) Rosenbaum. 2010. Design of Observational Studies: Ch 6
Recommended Reading:
1) Collier, Brady, and Seawright. Rethinking Social Inquiry. p 13-44 (Guidelines:
Summarizing DSIs Framework)
Chapter 13 (p229-233, 244-250, 252-266)
Appendix (p267-271)
2) Glossary from Gerring 2012
3) Brady, Henry E, D Collier, and J Seawright. 2006. Toward a pluralistic vision of
methodology. Political Analysis 14(3): 353368.
4) Hglund, Kristine and Magnus berg. 2011. Chapter 1, Doing Empirical Peace
Research. In Kristine Hglund and Magnus berg., eds., Understanding Peace
Research: Methods and Challenges.
5) Hglund, Kristine and Magnus berg. 2011. Chapter 11, Improving Information
Gathering and Evaluation. In Kristine Hglund and Magnus berg., eds., Understanding
Peace Research: Methods and Challenges.
Introduction: Theory Formation, Concepts and Different Methods
Required Readings:
1) Kellstedt and Whitten: Chapter 2, p22-44
2) Brady, D. 2003. Rethinking the Sociological Measurement of Poverty. Social Forces,
81(3):715-751.
3) Rosenfeld, J. 2010. The meaning of poverty and contemporary quantitative poverty
research British Journal of Sociology
4) Piazza, James A. (2011) Poverty, minority economic discrimination, and domestic
terrorism Journal of Peace Research, 48(3):339353
Recommended Readings:
1) Gerring, John. 2012. Social Science Methodology: A Unified Framework, Chapter 3
(Arguments), Chapter 6 (Concepts)
2) Krishna, A. 2010. Who Became Poor, Who Escaped Poverty, and Why? Developing and
Using a Retrospective Methodology in Five Countries, Journal of Policy Analysis and
Management, Vol. 29, No. 2, 351372 (2010)
3) Brady, D. Fullerton and Moren 2009. Putting Poverty in Political Context: A Multi-
Level Analysis of Adult Poverty across 18 Affluent Democracies, Social Forces, Volume
88, Number 1, September 2009, pp. 271-299
Case Studies and Qualitative Methods: Comparative Case Studies: Strengths,
Design Operationalisations
Required:
1) Kellstedt and Whitten: Chapter 3-4
2) George, Alexander L. and Andrew Bennett. 2004. Case Studies and Theory
Development in the Social Sciences. MIT Press.
3) Chapter 1 from Hedstrm, Peter and Swedberg, Richard. 1998. Social Mechanisms:
An Analytical Approach to Social Theory. Cambridge University Press
4) Gerring, John and Seawright, Jason. 2008. "Case-selection Techniques in Case Study
Research: A Menu of Qualitative and Quantitative Options" Political Research
Quarterly 61:294-308
5) Gerring, John. 2007. "Is There a (Viable) Crucial-Case Method?" Comparative
Political Studies 40:231-53
6) Gerring, John. 2005. "Causation: A Unified Framework for the Social Sciences."
Journal of Theoretical Politics 17:163-98
7) Gerring, John. 2004. "What is a Case Study and What is it Good For?" American
Political Science Review 98:341-54
Case Studies and Qualitative Methods: Case Studies: Process Tracing & QCA
Required:
1) Bennett, Andrew. 2010. Process Tracing and Causal Inference. In Henry E. Brady and
David Collier, eds., Rethinking Social Inquiry: Diverse Tools, Shared Standards, 2nd
edn. Lanham, MD. Rowman & Littlefield.
2) Seawright, J. 2005. Qualitative Comparative Analysis vis--vis Regression Studies in
Comparative International Development, 40(1):3-26.
3) Rihoux, B. 2006. Qualitative Comparative Analysis (QCA) and Related Systematic
Comparative Methods: Recent Advances and Remaining Challenges for Social Science
Research, International Sociology 21: 679-706.
Recomended:
1) Mahoney, J. 2010. Review Articles. After KKV.The New Methodology of
Qualitative Research World Politics 62(1):120-147
2) Fearon, J. 1991. Counterfactuals and hypothesis testing in political science. World
Politics.
3) Gerring 2011, Chapter 12 Generalizing the framework
Theory Formation & Formal Theory: Theory Formation and Inference
Required Readings:
1) Kellstedt and Whitten: Ch2
2) Gerring 2012: Ch 3 Argument, Ch 5 Concepts
Theory Formation & Formal Theory: Critical Approaches and the Philosophy
of Science
Required Reading:
1) Ch1 and 13 of Moses, Jonathon and Torbjrn L. Knutsen. 2007. Ways of Knowing:
Competing Methodologies in Social and Political Research.
2) Ch 7 of George and Bennett.
3) McGovern, Mike. 2011. Popular Development EconomicsAn Anthropologist among
the Mandarins. Perspectives on Politics 9(02): 345355.
Recommended Readings
1) Chapter by James Fearon and Alexander Wendt entitled Rationalism v. Constructivism:
A Skeptical View, in Carlsnaes, Walter, and Beth A Simmons. 2002. Handbook of
International Relations.
2) Latour, Bruno. The Last Critique. Harpers Magazine. April 2004.
3) Ch 1 of Gross, Paul R. and Levitt, Norman. 1994. Higher Superstition: The Academic
Left and Its Quarrels With Science.
Theory Formation & Formal Theory: Formal Theory/Game Theory
Required Readings:
1) p24-39 (The Modeling Enterprise) in Powell, Robert. 1999. In the Shadow of Power.
2) Preface (xiii-xv) in Gintis, Herbert. 2009. The bounds of reason: game theory and the
unification of the behavioral sciences.
3) Osborne, Martin J. 2004. An introduction to game theory.
Introduction (p1-8)
Skim Ch 2: 10-52
3.3 Electoral Competition
(Optional: Ch 5 for extensive form games)
Recommended Readings:
1) Harrington. Games, Strategies, and Decision Making.
2) Morrow, James. Game Theory for Political Scientists
3) McCarty and Meirowitz. Political Game Theory: An Introduction.
4) Fudenberg and Tirole, Game Theory.
5) Paul Krugman on how he works (listen to the gentiles, question the question, dare to be
silly, simplify simplify): http://www.princeton.edu/~pkrugman/howiwork.html
6) Gintis, Herbert. Game Theory Evolving
Theory Formation & Formal Theory: Applications and Critiques of Formal
Theory
Required:
1) Walt, S. M. 1999. Rigor or rigor mortis?: Rational choice and security studies.
International Security. 23(4).
2) Powell, R. 1999. The modeling enterprise and security studies. International Security.
24(2).
3) Zagare, F. C., and B. L. Slantchev. 2010. Game Theory and Other Modeling
Approaches.
Recommended:
1) D. Green and I. Shapiro, Pathologies of Rational Choice Theory, Chapter 1-3
2) The rest of the debate in International Security, Volume 24, Issue 2.
3) Aldrich, John H. and Arthur Lupia. 2011. Experiments and Game Theorys Value to
Political Science. in James N. Druckman et al., eds., Cambridge Handbook of
Experimental Political Science.
4) Paul Krugman on Formalism: http://web.mit.edu/krugman/www/formal.html
5) Paul Krugman on Formal Theory in Development Economics: http://www.pkarchive.org/
theory/dishpan1.html
Statistical Comparisons: Descriptive statistics
Required:
1) Kellstedt and Whitten, chapter 5 Measurements, chapter 6 Descriptives
Recommended:
1) Gerring 2011, Chapter 5 Descriptive Arguments, Chapter 7 Measurements
2) Adcock, R. and Collier, D. 2001. Measurement validity: A Shared Standard for
Qualitative and Quantitative Research The American Political Science Review,
93(3):529-546
Statistical Comparisons: Inferential Statistics & Bivariate Relationships
Required:
1) Kellstedt and Whitten, chapter 7 Statistical Inference, chapter 8 Bivariate
Hypothesis Testing
Recommended:
2) Gerring 2011, Chapter 8 Causal Arguments, Chapter 9 Causal analyses,
Statistical Comparisons: Introduction to Stata
Readings links:
http://www.ats.ucla.edu/stat/stata/default.htm (how to learn Stata)
http://statcomp.ats.ucla.edu/stata/ (text-book examples)
http://www.ats.ucla.edu/stat/dae/ (data analysis examples)
http://www.stata.com/links/resources1.html (general links for learning more about Stata)
Multivariate Relationships: Selection on Observables
Required:
1) Morgan, Stephen, and Chris Winship. 2007. Counterfactuals and Causal Inference:
Methods and Principles for Social Research.
Intro (except 1.3), Chapter 3 (skip the technical parts)
Recommended:
1) Morgan and Winship, Ch 6
2) Gerring 2012 Ch 9
Multivariate Relationships: Crosstabs and Regression Analysis
Required:
1) Kellstedt and Whitten, Ch 9 Bivariate Regression Models, Ch 10 Multiple
Regression Models I: The Basics
2) Alford, John R., Carolyn L. Funk, and John R. Hibbing. 2005. "Are Political
Orientations Genetically Transmitted?" American Political Science Review 99 (2, May):
153-168 (bivariate correlations, z-score tests for significance in differences between
correlations)
3) Achen, Christopher. 1977. Measuring Representation: Perils of the Correlation
Coefficient. American Journal of Political Science 21 (4): 80515.
Recommended:
1) Achen, Christopher H. 1982. Interpreting and Using Regression. London: Sage
Publications
2) Berry, William D. 1993. Understanding Regression Assumptions. London: Sage
Publications
3) Lewis-Beck, Michael. 1980. Applied Regression An Introduction. London: Sage
Publications
Multivariate Relationships: Regression Analysis
Required:
1) Kellstedt and Whitten, Ch 11 Multiple Regression Models II: Crucial Extensions
2) Krueger, James S. and Michael S. Lewis-Beck. 2007. Goodness-of-Fit: R-Squared, SEE
and Best Practice. The Political Methodologist 15 (1): 24
Multivariate Relationships: Logistic Regression & Interaction Effects
Required:
1) Brambor, Thomas, William Roberts Clark, and Matt Golder 2006. Understanding
Interaction Models: Improving Empirical Analyses , Political Analysis 14(1): 63-82
Recommended:
1) de Rooij, Elaine 2011. Patterns of Immigrant Political Participation: Explaining
Differences in Types of Political Participation between Immigrants and the Majority
Population in Western Europe. European Sociological Review (online, doi: 10.1093/esr/
jcr010 )
2) Stolzenberg, Ross, Maty Blair-Loy, and Linda J. Waite. 1995. Religious Participation in
Early Adulthood: Age and Family Life Cycle Effects on Church Membership. ASR
60:84-103 [very good probit article with graphs]
3) McCarty, J., Clark McPhail, and Jackie Smith. 1996. Images of Protest: Dimensions of
Selection Bias in Media Coverage of Washington Demonstrations 1982 and 1991. ASR
61:478-499 [nice odd-ratio interpretation]
Multivariate Relationships: Visualizing Data, Diagnostics
Required:
1) Kastellec, JP, and EL Leoni. 2007. Using graphs instead of tables in political science.
Perspectives on Politics 5(04): 755771.

Recommended:
1) Cleveland, William S.; Persi Diaconis; and Robert McGill. 1982. Variables on
Scatterplots Look More Highly Correlated When the Scales are Increased. Science. 216:
1138-1141
2) Cleveland, William S. and Robert McGill. 1987. Graphical Perception: The Visual
Decoding of Quantitative Information on Graphical Displays of Data. (with discussion)
JRSS A. 150: 192-229.
Design-Based Inference: Experiments and Natural Experiments
Required:
1) Dunning, Thad. 2010. Design-Based Inference: Beyond the Pitfalls of Regression
Analysis? Ch 14 in Brady, Collier, and Seawright. Rethinking Social Inquiry. 2nd
Edition
2) Rosenbaum. 2010. Design of Observational Studies.
Sections: 1-1.5
Sections: 5.1-5.2.4 (skim)
Summary: Key Elements of Design
3) Gerring 2012, Ch 10 (Causal Strategies)
Recommended:
1) Dunning, T. 2008. Improving Causal Inference. Political Research Quarterly 61(2):
282.
2) Angrist, JD, and JS Pischke. 2010. The Credibility Revolution in Empirical Economics:
How Better Research Design is Taking the Con Out of Econometrics. Journal of
Economic Perspectives 24(2).
3) Rosenzweig, M, and Kenneth I Wolpin. 2000. Natural Natural Experiments in
Economics. Journal of Economic Literature 38(4).
Design-Based Inference: Instrumental Variables, RD, Mechanisms
Required Reading:
1) Rosenbaum 2010, 5.3-end (skim)
2) Morgan and Winship, ch 8 (skim)
3) Acemoglu, D, S Johnson, and JA Robinson. 2001. The colonial origins of comparative
development: An empirical investigation. American Economic Review 91(5): 1369
1401.
Recommended Reading:
1) Morgan and Winship, Ch 7
Design-Based Inference: Multiple-Comparisons Bias, Publication Bias,
Robustness
Required Reading:
1) Morgan and Winship, 5.4.2
2) Gelman, A, and D Weakliem. 2009. Of Beauty, Sex and Power: Too little attention has
been paid to the statistical challenges in estimating small effects. American Scientist
97(4): 310316.
3) Ioannidis, John P A. 2005. Why Most Published Research Findings Are False. PLoS
Medicine 2(8): e124.
4) Leamer, EE. 1983. Let's take the con out of econometrics. The American Economic
Review.
5) Mullard, Asher. Reliability of new drug target claims called into question. Nature
Reviews Drug Discovery 10: 643-644.
6) Freedman, David H. 2011. Lies, Damned Lies, and Medical Science. The Atlantic.
7) Seife, Charles. The Mind-Reading Salmon: The True Meaning of Statistical
Significance. Scientific American.
Recommended Reading:
1) Blattman, Chris. 2011. Behavioral Economics and Randomized Trials: Trumpeted,
Attacked and Parried. Blog Entry.
2) Goldacre, Ben. 2011. Backwards Step on Looking into the Future. The Guardian.
3) Zimmer, Carl. 2011. Its Science, but Not Necessarily Right. The New York Times.
Research Design Assignment
MA Social Science Methods, 2011, Uppala University
Convenors: Katrin Uba and Allan Dafoe


One cannot repair a weak research design with a strong data analysis.
-Richard Berk
Research design begins, and ends, with the evaluation of plausible rival hypotheses.
-John Gerring, 2012
1


The basis of good research is a solid research design. Furthermore, methods are often more fun
and easy to learn when they are in the context of advancing ones research. For these reasons, the
major graded assignment in this course is a research design assignment in which you will
propose a design to study a question of interest. You should think of this as a draft proposal for
your MA thesis project or a grant proposal.

The first draft is due Friday Dec 9th
th
, the final draft due noon Jan 9
th
. The assignment should be
no more than 5 pages, including figures and tables (Times New Roman, 12 point font, 1.5 lines,
1 inch margins). Your list of references are not included in the 5 pages. Note, you should easily
be able to fill 40 pages with your thinking about your research design; being forced to articulate
it in only 5 pages is a challenge, not a break.
2
Submit your assignment as a pdf-format
3
with
filename consisting of RDA (for research design assignment), an underscore, and your last name,

1
In other words (Gerring 2012): A good research design allows one to prove the main
hypothesis and reject plausible alternatives.

2
Quoting Gerring 2012 on the arduousness of concision: Pascal once apologized to a
correspondent: The present letter is long, as I had no time to make it shorter.
3
Professionally, you should always submit documents as at least a pdf-format since it is more
robust, open, and free of viruses than proprietary formats like word.
like this: RDA_Smith.pdf. Post it to your groups folder in Studentportalen. If you compose your
assignment in word, also post a word-format of the assignment, similarly labelled.

The assignment should include a research question, primary and alternative theories with
precisely articulated rival testable hypotheses, and the research design (how you will test these
hypotheses, the data you will use, your case selection strategy, how you will analyse the data).
Your assignment should begin with a very short (e.g. 150 words) abstract summarizing your
research proposal.

You can choose any interesting and relevant research question, but you have to argue for this
choice. We do not expect you to have knowledge of all the relevant theories and prior literature
about your chosen topics, but you need to articulate at least two plausible theories that imply
competing hypotheses (otherwise, how could you draw any inferences!). These theories and
hypotheses may be drawn from the literature, may be your own synthesis of different literatures,
or may be your own ideas. You can use course materials as well as external literatures to argue
for the value of your research question, theory, and research design. At some point, you should
articulate precisely what your testable hypotheses are by stating, e.g., Hypothesis 1: ...

You can receive VG, G or U for this assignment. An excellent paper has all the following done
very well:
Short and appropriate title (either informative or intriguing)
Abstract summarizing proposal in less than one half of a page.
Clearly articulated research question. An argument for the importance of studying this
question.
Brief discussion of at least one good theory (scoring high on Gerrings criteria) that speak
to this research question, and ideally multiple.
Explicitly stated testable rival hypotheses.
4

A research design that can generate evidence that will discriminate between these
hypotheses.

4
One of the hypotheses can be the typical null of no difference.
Discussion of potential issues in the research design, and how they might be addressed
(case selection, confounding, selection biases, low power, dependence, limited access to
data, measurement issues, etc)
Appropriate citations to the literature, with a reference list at the end of the proposal (not
counted in the 5 pages).

You will turn in two drafts of this assignment. The first will be due by 5pm Friday December
9th
th
. You will get feedback on this draft in the seminar on December 16
th
. The second and final
draft will be due at noon January 9
th
. You will be graded on this final draft.

Below follows the (slightly edited, but otherwise verbatim) description of a similar assignment
by John Gerring, which you may use for additional insight into how to think about this
assignment. Gerrings assignment is geared towards PhD students, though, and the assignment is
much more extensive. You are not expected to, for example, provide a very thorough literature
review. Where there is other disagreement between Gerrings assignment and ours, clearly you
should use our description as your guide.

***

The assignment

The major written work for this class consists of a research proposal on a subject of your own choosing.
This proposal should take approximately the same form as a dissertation prospectus or grant
proposal. Indeed, you may consider this assignment as a dress-rehearsal for your masters thesis,
dissertation, or grant proposal. It should include a big theory (what its all about; the theoretical interest),
a specific hypothesis or set of related hypotheses, and a research design (how you propose to investigate
your hypothesis). Be as clear and well-organized as possible. Anticipate possible objections.
In addition to this brief set of guidelines, you are well-advised to consult various sources on
writing and publishing listed in the Addenda of the course syllabus.

OBJECTIVES
1. The theory, and the hypothesis, informing the study must be fairly general in scope. At the very
least, it must be broader than a single country. You may, however, focus in on a small terrain for purposes
of testing your idea. Sometimes, small samples have high external validity.
2. You are strongly encouraged to make a causal argument, rather than a predictive or descriptive
one. Predictive arguments may flow from causal arguments (indeed, they may be unavoidable), but they
would not typically be the main subject of a political science proposal. The reasons for preferring causal
over descriptive arguments are more complicated and should be briefly reviewed. First, descriptive
inference is in some respects harder (as discussed in the course). Second, we will be talking mostly about
causal arguments during the course of the semester. Third, the discipline is obsessed with causal
arguments, so it is a good idea to figure out how they work. And finally, there is more pay-off to you (on
the job market or wherever you end up).
3. You must propose a specific hypothesis. Clarify, if it is not entirely clear, what change on X is
predicted to result in what change in Y. Of course, you may be unsure about which of several possible
hypotheses to focus on. It is natural to begin a dissertation with a high degree of uncertainty. However, it
is not possible to write a convincing proposal by merely stating a series of questions. Exploratory
proposals are possible, but only if there are some plausible expectations that render the proposal
interesting theoretically and/or substantively. The more specific you can be, the better, for without such
specificity it is very difficult to engage questions of method the primary purpose of the course.
4. A dissertation is a large piece of work so there is space to make more than one argument. The
proposal, however, is a very short piece of work and there is space for only one main hypothesis, or a set
of hypotheses that are tightly integrated. Do not suppose that the proposal must incorporate all that you
will deal with in the dissertation (and the eventual book or set of articles that you plan to write).
5. Remember that you need not stick with your chosen theory and hypothesis through the rest of
your graduate career. This is an exercise, not a final product. Its purpose is largely heuristic, that is, to
help you think through the process of conceptualizing and implementing research and, more
specifically, writing a dissertation. It does not matter to me whether you do end up doing what you say
you will be doing. Consider the proposal a hypothetical plan of action. It matters whether this plan is
workable, but it does not matter if you choose to abandon it or dramatically reformulate it in future years.
6. The literature review must be extensive enough to show the value-added of your suggested
project. That is, you need to verify whether your idea has already been explored by other scholars, and if
so with what results. If your contribution is empirical rather than theoretical, then you need to show that
your research design is better than or adds something significant to the body of extant empirical work
on the subject. Your review should involve printed sources (published books and articles, as well as
unpublished papers) but also direct contact with scholars working in the chosen subfield. Sometimes, the
latter is the best way to arrive at a determination of whether a topic is truly novel, or merely
commonsense, and whether it is workable. But if not (and given the specialization of the academy, this is
unlikely) you should consult scholars by email wherever they happen to be. Remember that your prof in
this course is not an expert in everything (some might argue that he is an expert in nothing). The most that
I can do is to weigh in on the methodological components of your proposal; its substantive contribution is
probably not an area in which I will have much to say (and if I do say anything you should take it with a
grain of salt).

ORGANIZATION
As a summary of your proposal, please include this information on the first page, along with your name,
the title of the project, and the date:
1) Theory:
2) Hypothesis:
3) Research Design:
This is the format employed by AJPS in all their articles and you might want to scan this journal if you
are unsure what these categories mean or how they can be answered in several sentences. Of course, they
are highly reductive. You will have plenty of chance to explain if your proposal does not fit neatly into
these boxes.
In the body of the proposal, you might consider the following organization (keeping in mind that
this will vary somewhat according to the topic, the state of the literature on this topic, and your
argument):
Introduction. Introduce the general topic or question of your research. Whats the big picture?
You should say something about the everyday or policy significance of the topic if not this is not
apparent.
General theory and literature review. Clarify what the value-added of your study might be,
relative to extant work on the subject. There are three ways of establishing this. You may point out that a)
this topic is insufficiently studied; b) there are important unresolved questions (debates); or c) the
prevailing wisdom on this matter is wrong. (These three tacks are not mutually exclusive.)
Do not write pages and pages of literature review. Try to be as concise as possible, while
remaining comprehensive, in your review. The best way to do this is usually to structure your discussion
by way of substantive points, citing authors as you go. For example, rather than reviewing what Smith
(1980), Jones (1999), and Hall (2005) have to say, seriatim, disaggregate the literature on the topic by its
substantive findings and/or methods. For example: There are three approaches to the question of the
democratic peace: a) the case study (e.g., Smith 1980), b) the crossnational statistical study (e.g., Jones
1999), and c) the formal model (e.g., Hall 2005).
If the literature on your subject is vast and complicated, you might consider presenting them in
tabular format. E.g.,

Study Finding Sample Method Weakness
Smith (1980) Positive Switzerland Case study
Jones (1999) Negative All countries, 1960- Large-N Xnatl
Hall (2005) Positive None Formal model

Hypothesis. Sometimes, the hypothesis can be stated as part of the general theory. Sometimes, it
is helpful to introduce it later, as an instantiation of that theory.
In any case, if your argument is rather complicated, draw a diagram showing how the major
factors in your theory inter-relate. As an example, here is a diagram that I constructed for a recent project
on democracy and development:


1. Economic policy
2. Infrastructure
3. Policy continuity
4. Social peace
5. Environmental policy
6. Education
7. Public health
8. Gender equality
9. Economic growth

1. Policy investments
2. Learning
3. Institutionalization
4. Inclusion
5. Consensus, Stability

Causal
pathways

Outcomes

Democratization
Causal
factor



Research design. Next, discuss the nature of the evidence that you will be evaluating and the
form of analysis that you will employ. Since this is the main topic of the course, this is the section that I
will be paying closest attention to.
If there are a relatively small number of cases and a large number of variables, I strongly
encourage you to construct a truth table in which you score each case (or each case type) on each
dimension (i.e., on each independent and dependent variable). This will allow you and us to evaluate
the evidence in a concise format. Of course, this is not possible in projects that incorporate a large sample
or where the scoring of cases is unknown, as with experiments.
Be sure you justify your choice of research design. How does your approach differ (or not) from
other writers? Why did you choose this research design, and not others?
Finally, and very importantly, discuss the possible weaknesses of this research design. Recall that
the objective of this course is to teach methodology, not simply to develop good research. This means
enhancing methodological selfconsciousness. If your research design has flaws or limitations (as all do),
acknowledge these. Your job is not to identify a perfect research design but rather the one that is best
possible, under the circumstances given limited time, resources, access to materials, ethical constraints,
and so forth.