Thesis: thinking quantitatively changes the way we frame and answer questions in ways we often don't notice. James Robinson, of Acemoglu and Robinson fame (ie, Why Nations Fail, @whynationsfail, Colonial Origins; Reversal of Fortune, and so forth), gave a talk at Princeton last week. It was a good talk, mostly about Why Nations Fail. My main thought during his presentation was that it's simply very difficult to develop a parsimonious theory that covers something as complicated as the long-term economic and political development of the entire world! As Robinson said (quoting someone else), in social science you can say "more and more about less and less, or less and less about more and more."
The talk was followed by some great discussion where several of the tougher questions came from sociologists and political economists. I think it's safe to say that a lot of the skepticism of the Why Nations Fail thesis is centered around the beef that East Asian economics, and especially China, don't fit neatly into it. A&R argue here on their blog -- not to mention in their book, which I've alas only had time to skim -- that China is not an exception to their theory, but I think that impression is still fairly widespread.
But my point isn't about the extent to which China fits into the theory (that's another debate altogether); it's about what it means if or when China doesn't fit into the theory. Is that a major failure or a minor one? I think different answers to that question are ultimately rooted in a difference of methodological cultures in the social science world.
As social science becomes more quantitative, our default method for thinking about a subject can shift, and we might not even notice that it's happening. For example, if your main form of evidence for a theory is a series of cross-country regressions, then you automatically start to think of countries as the unit of analysis, and, importantly, as being more or less equally weighted. There are natural and arguably inevitable reasons why this will be the case: states are the clearest politicoeconomic units, and even if they weren't they're simply the unit for which we have the most data. While you might (and almost certainly should!) weight your data points by population if you were looking at something like health or individual wealth or well-being, it makes less sense when you're talking about country-level phenomena like economic growth rates. So you end up seeing a lot of arguments made with scatterplots of countries and fitted lines -- and you start to think that way intuitively.
When we switch back to narrative forms of thinking, this is less true: I think we all agree that all things being equal a theory that explains everything except Mauritius is better than a theory that explains everything except China. But it's a lot harder to think intuitively about these things when you have a bunch of variables in play at the same time, which is one reason why multiple regressions are so useful. And between the extremes of weighting all countries equally and weighting them by population are a lot of potentially more reasonable ways of balancing the two concerns, that unfortunately would involve a lot of arbitrary decisions regarding weighting...
This is a thought I've been stewing on for a while, and it's reinforced whenever I hear the language of quantitative analysis working its way into qualitative discussions -- for instance, Robinson said at one point that "all that is in the error term," when he wasn't actually talking about a regression. I do this sort of thing too, and don't think there's anything necessarily wrong with it -- until there is. When questioned on China, Robinson answered briefly and then transitioned to talking about the Philippines, rather than just concentrating on China. If the theory doesn't explain China (at least to the satisfaction of many), a nation of 1.3 billion, then explaining a country of 90 million is less impressive. How impressive you find an argument depends in part on the importance you ascribe to the outliers, and that depends in part on whether you were trained in the narrative way of thinking, where huge countries are hugely important, or the regression way of thinking, where all countries are equally important units of analysis.