Additional Guidelines on the Methods Paper

Our expectations for the methods paper are that the author (a) demonstrate competence with methodological technique(s) that go beyond the standard toolkit used in typical quantitative analyses or modeling and (b) demonstrate a self-consciousness about the methodological choices made and their implications.

Criterion (a) does not imply that a paper needs to invent a completely new approach or technique; however, it typically should make use of relatively advanced technique(s) and should discuss their methodological underpinnings. Though there are no hard-and-fast rules, this means that a quantitative analysis would probably need to go beyond OLS regression or a standard limited dependent variable model (logit, probit, etc.), and a modeling paper would need to do more than apply a standard, off-the-shelf model to a slightly different context or problem.

Criterion (b) can be met in several ways; here we are interested in seeing that the author can think through and make methodological choices that further the ability to make inferences. This might involve a review and explanation of several approaches that solve a particular methodological problem, along with a discussion of why one is best for the problem at hand. What is at stake—methodologically and substantively—in the choice of model or estimator? Why is the problem this approach solves an important one to solve (especially compared with other problems that might be solved)? How does this choice give us better methodological leverage on the question?

Each paper is, of course, unique, and each paper will place greater focus and effort on one or the other of the two criteria. Methods papers should be innovative. The innovation need not be purely methodological in the sense that it would be a candidate for publication in Political Analysis. Rather, the innovation might consist in the application of a somewhat advanced or novel technique or approach to a new context as long as the paper focuses clearly on (1) the rationale for the choice to use this approach and (2) on what we learn that is new. Thus, for example, a paper on measurement could develop an incremental advancement in the IRT model to solve a particular problem (which would be leaning more on criterion “a”); or by executing a more standard IRT while explaining clearly the mathematics of IRT (criterion “a”) and how this improves—in detail and substantively—over prior/other/simpler approaches (criterion “b”).

In order for a game theory paper to meet the standard of “competent” and “self-conscious”, the paper should include a fully specified game and solutions, possibly including comparative statics. All propositions should be proved, and the game should be appropriately advanced (for example, the model should include parameters rather than numbers, be a repeated game, or have incomplete information). Moreover, a good modeling paper is very substantively motivated and all the modeling decisions and assumptions are justified. The model does not have to be tested empirically with data. However, case studies illustrating the mechanism should be included if the game theory paper does not include an empirical test. Additionally, the findings of the model should be explained intuitively, highlighting the substantive contributions, in addition to being technically specified.

Finally, it is important to understand that a methods paper will look somewhat different from a “substantive” article manuscript. Specifically, it will devote considerably more attention to the methods used and to the connections between methodological choices and substantive inferences.