Old "Methodology/Theory" Debates
I have decided to grant you guys a post where you can beat each other senseless over methodological/formal theory debates. Please keep them off the job rumors post. Also, I am back in town so I will be watching it and deleting individual attacks. Keep it civil!!
posted by American and Comparative Politics Job Blog at 6/19/2007 08:28:00 PM
907 Comments:
A debate? How totally whack that would be, yo!
From the old thread -
The formal models generate testable hypothesis that you can then test using empirical methods.
This is really not a good characterization of the utility of formal models. They don't need to be an adjunct of empirical work to be valuable. They are valuable in their own right even if they are never or can never be tested.
The McKelvey-Schofield 'chaos' results are a great example. Not really testable and they don't make predictions. But they clearly identify minimal conditions on what must be specified for a theory to pin down behavior at all.
"Not really testable and they don't make predictions"
As a friendly amendment, I would put it differently.
Purely formal results (such as the McKelvey-Schofield results, Arrow's Theorem, Gibbard-Satterthwaite, etc. are -true-
No tests necessary. The "proof" of a theorem is indeed proof of its validity.
I have heard tale told of a formal theorist being asked how he/she planned to "test" a theorem of his/hers. Of course, the proper response to that query is, "using the same seminal methodology used to prove that two plus two equals four."
RE: 11:29 p.m.
McKelvey-Schofield results, Arrow's Theorem, Gibbard-Satterthwaite, etc.
*********************************************
Do these relate to economic or political behavior? Or is the argument that they have implications for both?
The (universally true) theorems have to do with the properties of preference aggregation.
Suppose, for example, that you disagree with Aristotle and think democracy is a superior form of government because it guarantees that society's choices will be made by the "will of the people" and furthermore that such choices can never be manipulated by elites. To be specific, let us suppose that by "democracy" you mean "direct democracy by simple majority rule."
The purely theoretical results basically tell you that you cannot guarantee the properties you had ascribed to this form of "democracy." The theorems actually say much more than this, but this is one way in which formalization can be useful for normative theory.
However, this raises an interesting point. Theorems that are universally true are not explanations per se of what we observe about the world because they are not falsifiable (in the Popperian view of science). What is their place in a positive theory of politics, as opposed to a normative theory?
Do the normative concerns of social science make us different than, say, the physical sciences (who can focus on the positive and leave the normative aspects to philosophers and ethicists)? (I know of some well-known political scientists who do believe in this kind of strict separation, but I don't take that view.)
"Or maybe has to do with a lack of self-control. Of course, no one could ever model that."
Schelling anyone?
6/03/2007 3:35 AM
Sarcasm, dude. Chill out.
RE: 9:22 a.m.
Do the normative concerns of social science make us different than, say, the physical sciences (who can focus on the positive and leave the normative aspects to philosophers and ethicists)? (I know of some well-known political scientists who do believe in this kind of strict separation, but I don't take that view.)
*****************************************************************
I would certainly argue that the empirical subject of the social sciences, humans, renders normative/ethical concerns central to our endeavors as scientists. This could be why efforts to wring ethics and normative issues out the discipline (e.g., American Politics) has led in the minds of many to sterile and intellectually marginal political science work. Of course, a key reason why many in political science seek to "expel" normative considerations from their empirical research is because they are ostensibly trying to mimic the physical sciences.
As I have argued over the past month, seeking to model political science, or any social science, on the physical sciences is misguided, since first and foremost the physical sciences have something social scientists do not have – immutable laws to work with. Secondly, and this is new dimension to my argumentation, physical scientists must actively incorporate normative dimensions into their research if society is going to achieve a level of democracy and ethics that is necessary for the survival of humanity. The myth of a valueless science is a factor in the conversion of the planet into something that is inhospitable to humanity – alla global warming.
Thus, it is paramount in this period that scientists understand how/why they are helping to irrevocably change the world for the worse (from the perspective of humanity). During this critical period, the leading journals of the discipline of political science – the discipline with the greatest implications for normative considerations – are trying very hard to stay silence on normative/empirical matters.
"Do these relate to economic or political behavior? Or is the argument that they have implications for both?"
They are universally true, relying onno assumptions about behavior.
The G-S theorem is about voting mechanisms, essentially, and Arrow's theorem is about preference aggregation in -any- (very) minimally democratic system.
The preferences are completely abstract and thereby account for behavior in any social/cultural/economic/etc. realm.
McKelvey-Schofield is also true in any realm, and essentially states that the presumption that group preferences are aggregated by simple majority rule essentially buys you "nothing" at the aggregate level in terms of prediction. These results have been misinterpreted (in well-meaning ways), but are seminal nonetheless.
One can also add Nakamura's theorem to the list, as well as Austen-Smith & Banks's results regarding the Condorcet jury theorem (APSR, 1996).
"However, this raises an interesting point. Theorems that are universally true are not explanations per se of what we observe about the world because they are not falsifiable (in the Popperian view of science). What is their place in a positive theory of politics, as opposed to a normative theory?"
Well, this is an interesting question.
First, I'd submit that positive theory and normative theory _are_ very closely connected and both central to our identity as a distinct social science. (Yes, I typed "science.")
Second, positive theory, in the form of formal results, is central to empirical political science (and a "positive" science in general) because it establishes things such as
1) observational equivalence
2) impossibility results
3) logical implications following from (either empirically induced or normatively deduced) a priori assumptions.
I don't think that the "universal truth" of formal theory makes it superior to empirical work by any means -- they are two birds of slightly different feathers, hopefully headed in the same direction. :)
RE: 10:44 am
Your assertion that the field of American Politics is being wrung free of "ethics and normative issues" is false on its face. The various debates about (to name only a few) race, inequality, interest group representation, redistricting, direct democracy, etc., are ALL motivated by normative and ethical considerations.
Indeed, I would be willing to argue over 2 or 3 fingers of bourbon that -all- work in American politics essentially comes down to either
A) "who wins and who loses under institution X" -- this is clearly a question fraught with normative and ethical implications and overtones
or
B) "How does factor Y affect the (voting/contribution/participation/etc.) behavior by people in group Z?" Questions in this class may be posed in a manners that are less fraught with ethical and normative implications, but the answers and approcahes to questions in this class are inevitably relevant to those who care about ethical and normative questions dealing with the factor, behavior, and/or group(s) examined in the study.
I have to say, you're clearly not making any progress with the immutable law point, and repeating the refrain of "American politics is intellectually arid," etc., is not very likely to win you any additional adherents on this blog.
Instead of claiming that everything as it is currently done is wrong-headed, why don't you propose a clearly defined question and method for answering it? Perhaps you have published this argument in more detail -- if so, let me know where to find it. I don't think this bastardized blog format does your argument justice.
'In a 2005 article in the Journal of the American Medical Association, epidemiologist John Ioannidis showed that among the 45 most highly cited clinical research findings of the past 15 years, 99 percent of molecular research had subsequently been refuted. Epidemiology findings had been contradicted in four-fifths of the cases he looked at, and the usually robust outcomes of clinical trials had a refutation rate of one in four.
'The revelations struck a chord with the scientific community at large: A recent essay by Ioannidis simply entitled "Why most published research findings are false" has been downloaded more than 100,000 times; the Boston Globe called it "an instant cult classic." Now in a Möbius-strip-like twist, there is a growing body of research that is investigating, analyzing, and suggesting causes and solutions for faulty research.' (Seed Magazine
RE: 10:55 a.m.
Is this a set of logic theorems, as opposed to specific empirical claims? This is not to deny that such theorems can have implications for empirical/normative behavior/analysis.
Also, I will add that I am happy that the esteemed c.c. banana is contributing to this blog. =-)
Well, thank you 11:09am -- CC likes to keep it real, or at least "realer."
The theorems I have cited are not empirical. They are logical results, all essentially starting with some institution (or family of institutions satisfying some property or properties, such as the "method of majority rule," as characterized by May's theorem) and following the logical implications of these assumptions for the "possibilities" of such institutions. For example, Gibbard-Satterthwaite states, in very coarse and simplistic terms, that democratic decision-making and the incentive to vote strategically (or "insincerely," depending on how grumpy one is) necessarily walk hand-in-hand in any environment in which there are three or more alternatives from which a group must choose.
This, of course, does not prove that people vote strategically, nor does it prove that such behavior is "bad." (C.C. would argue (again, preferably with 2 or 3 fingers of bourbon) that such behavior is not bad...but I digress.)
My point with these (and other formal results) is that I hope and believe that our entire field can be proud of them. We study a very tough family of topics, and formal results clarify exactly *why* it is important for us to study the wide array of topics. For example, C.C. would argue (even without the bourbon) that Arrow's (Im)possibility theorem is the raison d'etre of our entire field -- particularly those dealing with political institutions. Individuals who believe that "culture matters" in constitutional design/democratization (as one example) can (and should) lean on Arrow's theorem as (a or the) central justification for their studies. Of course, I doubt that everyone would agree with this assessment, which is itself very "meta.")
"I have to say, you're clearly not making any progress with the immutable law point"
I agree. Someone needs to go read Popper. That's not to say that the logical/philosophical underpinnings of science haven't evolved since, but the point continues to undermine the credibility of any other claims s/he makes.
"2) impossibility results"
Are there analogues in the physical sciences for these kinds of results? Just curious.
Who is this c.c. banana? I wonder if I am related. Because I don't know, I'm starting to rethink the validity of my "common knowledge" assumption...
RE: 11:08 a.m.
I believe there is a conscious effort in the discipline (especially American Politics) to describe social/political phenomena as resulting from random atoms (i.e., people, groups, political parties) smashing into each other, and from this smashing arises patterns of behavior that political scientists then are expected to document. This is simply a false picture of U.S. society and politics. Instead, what must be actively engaged is the notion of class and class hegemony. More specifically, how class power actively and directly shapes political institutions and their behavior.
I will use the work entitled _The Politics of Air Pollution_ to demonstrate the analytical value of class analysis. As the name of the book suggests, it deals with the politics of air pollution. It does so in the U.S. context. Now, the assumption in the U.S. environmental politics literature, heretofore, is that U.S. clean air policies are the result of businesses interacting with environmental groups (hence, groups smashing into each other resulting in a pattern – technology to abate localized air pollution). As the empirical record demonstrates, this is a false conclusion. Firstly, efforts to abate localized air pollution predate the modern environmental movement by about 80 years. Secondly, these efforts sought to use technology exclusively to abate localized air pollution.
Instead of presuming smashing atoms to analyze U.S. air pollution politics, the author of _The Politics of Air Pollution_ employs class analysis. Utilizing this approach he was able to see that real estate interests, an important segment of the nation's economic elite, have an interest in abating localized air pollution. This is because excessive air pollution harms local land values. Moreover, technology is consistent with the economic and political interests of large land holders, because while technology can reduce localized air pollution it does not directly interfere with efforts to attract more and more investment and people to an area (i.e., local economic growth). Such growth generally push up land values.
This class analysis approach allows the author to acknowledge the long history of air pollution abatement campaigns, and, indeed, place them at the center of his analysis. Additionally, he can explain why technology was at the center of these campaigns. Finally, it allows him to explain why technology remains at the center of U.S. airborne emission abatement policies, even though technology has failed to provide clean air or abate greenhouse gas emissions.
RE: 11:40 a.m.
Someone needs to go read Popper.
********************************************************
Given that this is a methodology/theory blog, I am perfectly happy to delve into a discussion of Popper. If I have to go examine Popper's work in-depth in order to effectively participate in this discussion, I am more than will to do so.
My point is simply that if Popper has something to say of irrelevance, I would kindly ask that you lay out his thinking. Please do not be shy.=-)
"the author of _The Politics of Air Pollution_ employs class analysis. Utilizing this approach he was able to see that real estate interests, an important segment of the nation's economic elite, have an interest in abating localized air pollution"
Interesting. I wonder who the author is, and if anyone but the poster has read the book. Isn't it ironic that this illustration by the anti-formal, anti-rational choice, anti-American politics as it is currently practiced, anti-quantitative, anti-immutable law poster involves a significant amount of rational economic self-interest?
Rails against the mainstream, but when all is said and done is really no different. And a remarkable display of ignorance of both what science is and mainstream American politics.
RE: 12:26 p.m.
Isn't it ironic that this illustration by the anti-formal, anti-rational choice, anti-American politics as it is currently practiced, anti-quantitative, anti-immutable law poster involves a significant amount of rational economic self-interest?
*****************************************************************
To argue that economic self-interest plays a significant role in political issues is not a concession to rational choice, quantitative methods, or immutable laws of social/political behavior. It seems to me that all these presume that individual behavior can be precisely modeled (including probabilities), whereas the author of the book I referenced does nothing of the sort. Instead, he analyzes political movements by making broad references to economic interests.
RE: 12:26 p.m.
Assuming this person is a graduate student, is calling him/her a "clueless grad student" helpful? I would think we would want to encourage and educate, not insult, such a person.
12:26 here, I am the one that is clueless. I think counts as self-deprecation.
RE: 1:56 p.m.
True enough.
Anonymous said...
Paul:
What are some examples of formal theory "lite" that you've been successful with in your class? How well do the students take to them?
The syllabus is here: http://people.reed.edu/~gronkep/pol210-f06/index.html
I use Shepsle and Bonchek but it is severely limited w/o problem sets. I supplement that with self-designed problem sets. (If I ever find the time and a partner, perhaps I'll write a textbook of my own.)
They learn the basic technique for finding a Nash eq., they do some backwards induction, they do some basic spatial analysis. Some from Shepsle and Bonchek, some from Dixit and Skeath, some cribbed from Stewart's Congress book.
I use the paradox of turnout as my substantive "hook" in the class, first presenting the problem as one of rational decision making, then voting in the aggregate as a collective action problem, then institutions as a "solution" (Rosenstone and Hansen).
Then finally (whew!) they get baby statistics and use the sda online stats analysis system to write a paper analyzing turnout as d.v.
We also look at "scholarly" articles applying the techniques, including two pieces by Jeff Jenkins (spatial+history), a piece by Plous (application and extension of PD), Paolino et al. on third party voting.
As I say, it's a friggin' burdensome class and won't work if you have to teach many more than 24. That's even a heavy load.
But it does work, for me at least.
So here's a real story apropos of my last posting: a week after graduation, I ran into one of my students, and he excitedly told me: "The new d-nominate scores just came out. You should go look at the rankings for the new Democratic members of Congress."
Sick? Rewarding? Little of both?
Both, Paul, but you know you loved it either way.
RE: 11:08 a.m.
A) "who wins and who loses under institution X" -- this is clearly a question fraught with normative and ethical implications and overtones
*******************************************
Dear Prof. Banana,
You are right that such matters are fraught with normative and ethical implications and overtones. The difficulty is that the core of American Politics tries to avoid these implications and overtones as much as possible. In other words, if business defeats environmentalists, articles in _APSR_, _JOP_, or _AJPS_ treat the outcomes without exploring the implications of this defeat. The defeat is simply an intellectual curiosity, or, worse, a data point.
Moreover, in these journals there is ostensibly little interest in the structural advantages that big business, and its ownership, has over all interest groups, and, again, the normative implications of this. Is it any wonder that people feel the discipline cannot lead in of deepening normative or ethical considerations throughout society?
in these journals there is ostensibly little interest in the structural advantages that big business, and its ownership, has over all interest groups
This is totally false and would get you flunked on any decent field exam dealing with interest group politics in the US. Representation of citizen vs. business interests is a big issue in this literature.
But I think we all see the main issue after George's last post. We are not sufficiently committed to prosecuting an ideological agenda that is anti-business and pro-environment. If we had formal models proving that Halliburton is evil or produced an R plot of Dick Cheney holding a pitchfork, George would be applauding.
RE: 7:00 p.m.
We are not sufficiently committed to prosecuting an ideological agenda that is anti-business and pro-environment. If we had formal models proving that Halliburton is evil or produced an R plot of Dick Cheney holding a pitchfork, George would be applauding.
*****************************************************************
You can try to push it to the margins if you like, but how/whether business influence is leading to environmental destruction is a prime political issue. This is going to be the burning political issue of future generations (provided we have future generations), not whether party unity leads to greater incumbency turnover, for example. As was discussed in the last thread/blog, if the discipline is no of importance/interest to people outside of the discipline, it is because we are making so.
RE: 7:00 p.m.
Contrast the big three, to _Polity_, where environmental concerns are more throughly treated.
RE: 7:00 p.m.
This is totally false and would get you flunked on any decent field exam dealing with interest group politics in the US. Representation of citizen vs. business interests is a big issue in this literature.
****************************************************************
I beg to differ. I pursue the literature on this subject rather aggressively. There is shockingly little on big business political influence in the journals of _APSR_, _JOP_, or _AJPS_. One exception over the last few years to this is:
Yackee, Jason Webb and Susan Webb Yackee. 2006. "A Bias Toward Business? Assessing Interest Group Influence on the U.S. Bureaucracy." _Journal of Politics_ 68, no. 128-139.
If someone knows of business influence articles I am overlooking, please bring them to my attention.
"...not whether party unity leads to greater incumbency turnover..."
The point of much of the institutional and historical literature on Congress is that the source(s) of party (dis)unity have changed over time and have large implications for the legislative treatment of, for example, antitrust, environmental, and civil rights issues. Of particular interest is the role that incumbency turnover in both the environmental and civil rights policy arenas. In fact, I will respectfully (score!) submit that, if you think about the history of congress and the subtance of the modern scholarly congressional literature, you will see that the fates of environmental policy (as played out from about 1984-current day) and Federal civil rights legislation (mid 60s to early 70s) are inextricably linked -precisely- through the subtle and yet nonetheless ever-present links between partisanship "on the floor" and constituency responses.
Similarly, one can not understand the origins and development of the present relationship between corporate/business interests and members of Congress without understanding the dynamics of partisanship, procedure, and seniority within the Democratic party from 1960-1976.
I am sympathetic to incorporating ethical, normative, and policy concerns as far as possible within our research, but it is completely unfair for you to lob around charges that no one does so. I would submit that more scholars do so (even those who "publish in the evil Big 3") than do not.
I am not, however, in the spirit of honesty, convinced that environmental degradation will be the "burning issue" of the future. Our perception of what will be important in the future is quite unreliable. For example, in the late 19th century, very wise observers worried that the next big crisis would be the exhaustion of the world's coal supplies.
Paul:
What readings/articles go over especially well (or not so well) in your class? How interested do the students get? I would assume that this stuff isn't what they imagined when they signed up for a Political Science class.
Why does it seem to me that some Perestroikans believe political scientists should behave as if they are policy advocates? That Political Science would be best served if all "research" would be along the lines of what appears in Foreign Affairs? Or perhaps Mother Jones.
RE: 7:54 p.m.
In fact, I will respectfully (score!) submit that, if you think about the history of congress and the subtance of the modern scholarly congressional literature, you will see that the fates of environmental policy (as played out from about 1984-current day) and Federal civil rights legislation (mid 60s to early 70s) are inextricably linked -precisely- through the subtle and yet nonetheless ever-present links between partisanship "on the floor" and constituency responses.
Similarly, one can not understand the origins and development of the present relationship between corporate/business interests and members of Congress without understanding the dynamics of partisanship, procedure, and seniority within the Democratic party from 1960-1976.
*****************************************************************
I would kindly submit that you have to make a more explicit case of how such maneuvering/dynamics impacted the outcome of environmental policy in the U.S. Because the reality is that both major parties throughout the post-World War II era have been content to relegate environmental issues to secondary/tertiary status and dealt with constituency environmental concerns through rhetoric and legislation that is severely under-funded. These issues are given scant attention in the leading journals of the discipline.
Finally, your effort to minimize the crisis of global warming is disconcerting. It demonstrates a profound lack of knowledge on the issue – perhaps telling of the importance the discipline assigns this grave matter. The International Panel on Climate Change (representative of the world's climate scientists) recently concluded that we have eight years to effectively address our greenhouse emissions before we create irrevocable, catastrophic, and unmanageable global temperature increases.
I beg to differ. I pursue the literature on this subject rather aggressively. There is shockingly little on big business political influence in the journals of _APSR_, _JOP_, or _AJPS_.
Ah, the ol' base rate neglect rears its head again.
Anyway your filter is too narrow. This stuff is part of the groups literature from the upper class accent in the pluralist heaven on down.
Finally, your effort to minimize the crisis of global warming is disconcerting. It demonstrates a profound lack of knowledge on the issue
Man we have hit the nail on the head with GAG's motivation for all this. Disagree with his politics? You're uninformed. Decline to emphatically agree with his politics? Uninformed.
GAG's a good liberal and formal and statistical methods simply do not force a political persuasion on you. All this universal law nonsense is just window dressing for that. It's a political objection plain and simple.
formal and statistical methods simply do not force a political persuasion on you.
------------------
NO you're WRONG. Because they don't force reactionary liberal politics on you they allow you to be a radical conservative. They force a persuasion, a persuasion of LIES.
Tell me, do you recommend Dick Cheney to referee your work at the APSR? And did you know that Bush lied and people died?
I'm sure all the liberal formal/quant types out there will be set straight now that they've read 8:42's post.
One aspect of this discussion that is interesting is the reaction to the suggestion that the discipline, and especially its leading journals, should focus more attention to the linkages between business political influence and environmental destruction. I am specifically referring to the claims that by focusing on the business role in environmental politics the outcome will be advocacy – a pro-environmental group bent. Apparently, the individuals who are putting forward these reactions do believe that big business is directly responsible for unsustainable environmental destruction. Hence, any exploration of the issue from a political angle will lead to unwelcome results.
"I would kindly submit that you have to make a more explicit case of how such maneuvering/dynamics impacted the outcome of environmental policy in the U.S. "
I would not-quite-as-kindly submit that I don't "have" to do anything of the sort. My points are well documented throughout the Congressional and bureaucratic literature, which I increasingly suspect that you don't follow very closely. The claim that environmental regulation has been historically underfunded is beside the point. U.S. environmental policy was dramatically affected by the changes within the Democratic caucus between 1968 and 1974. Please feel free to refute this claim. I will rest on the well-documented sequence of events behind the creation of the EPA, the passage of the Solid Waste Disposal Act of 1965, the
Resource Conservation and Recovery Act in 1976, and the variety of implementation problems that plagued the EPA - and arguably followed as a result of Congressional over-involvement in the policy area.
More generally, I wouldn't classify the National Environmental Policy Act of 1969 as anything less than of primary importance (i.e., definitely not "tertiary") in many different ways, affecting both congressional and bureaucratic politics in profound and lasting ways.
Similarly, the Clean Air Act and its subsequent amendments have been, um, pretty high profile. I don't know what your definition of politics is, but mine extends beyond a simplistic presumption that all issues should be discussed and fretted over publicly in direct proportion to (my perception of) "their importance."
Also, I didn't dismiss global warming -- I doubted that it will be the "burning issue" of the next generation. I'd place my money on things like, oh -- money, jobs, health care -- will be the burning issue.
I would also kindly submit that you should not refer to the Banana as being profoundly ignorant, respectfully reminding you that I am quite possibly the world's perfect fruit.
"I would also kindly submit that you should not refer to the Banana as being profoundly ignorant..."
I, for one, am impressed by the depth of th Banana's substantive and theoretical knowledge!
"...respectfully reminding you that I am quite possibly the world's perfect fruit."
Indeed, you are proof of the existence of God:
Defending The Existence of God With a Banana (YouTube video)
I beg to differ. I pursue the literature on this subject rather aggressively. There is shockingly little on big business political influence in the journals of _APSR_, _JOP_, or _AJPS_. One exception over the last few years to this is:
Yackee, Jason Webb and Susan Webb Yackee. 2006. "A Bias Toward Business? Assessing Interest Group Influence on the U.S. Bureaucracy." _Journal of Politics_ 68, no. 128-139.
If someone knows of business influence articles I am overlooking, please bring them to my attention.
6/03/2007 7:49 PM
-----------------------------
While much of it is not the journals you list, Gray and Lowery and various colleagues have a number of articles over the last decade about the dominance of institutions (by which they mean business interests in the sense of Salisbury).
Gordon and Hafer have a recent article (APSR?) about oversight of the nuclear energy industry.
If I think of any more or manage to publish some of my own, I'll let you know.
RE: 9:19 p.m.
Banana,
I love you (as a fruit), but how could you "claim that environmental regulation has been historically underfunded is beside the point." What good are regulatory laws if the agency responsible for formulating regulations and implementing them cannot so do because of lack of financing. In part because of lack of adequate funding, leading into 1990 the Environmental Protection Agency only set regulations for 8 airborne toxic emissions -- in spite of the fact that it was aware that well over 100 were actively being emitted by industry. A 2004 Inspector General report found that due to under-staffing and inadequate investment in computer equipment the EPA did not have the information needed to determine the impact of water pollution regulations, nor compliance with these regulations. As a result of this critical lack of data, the Inspector General's report asserted that the "EPA cannot support a statement made in its recent Annual Report that industrial discharges of pollutants have been reduced by billions of pounds as a result of [water] effluent guidelines."
You also wrote: "Also, I didn't dismiss global warming -- I doubted that it will be the 'burning issue' of the next generation. I'd place my money on things like, oh -- money, jobs, health care -- will be the burning issue." With a severe depletion in food production due to global warming, somehow I think that this issue will trump all others.
As for your point that: "More generally, I wouldn't classify the National Environmental Policy Act of 1969 as anything less than of primary importance (i.e., definitely not 'tertiary') in many different ways, affecting both congressional and bureaucratic politics in profound and lasting ways." From an environmental perspective, this law has had virtually no impact (see Matthew J. Lindstrom & Zachary A. Smith. 2001. _The National Environmental Policy Act: Judicial Misconstruction, Legislative Indifference & Executive Neglect._ College Station: Texas A&M University Press.)
I think the difficulty with your analysis of U.S. environmental politics and policy is that it is excessively focused on Congress and the legislative aspects of environmental policy. An accurate understanding of U.S. environmental policies can only be fully grasped by also examining the implementation/administration of such policies.
RE: 9:44 p.m.
Thanks!!!
"What good are regulatory laws if the agency responsible for formulating regulations and implementing them cannot so do because of lack of financing."
Good question. This is where it helps to step away from the details of policy and to think about how policies and their implementation are the result of politics. A number of previous posts hint at the relevant literature. (Did someone indirectly refer to Schattschneider?!)
"You also wrote: "Also, I didn't dismiss global warming -- I doubted that it will be the 'burning issue' of the next generation. I'd place my money on things like, oh -- money, jobs, health care -- will be the burning issue." With a severe depletion in food production due to global warming, somehow I think that this issue will trump all others."
Interestingly, this looks like a prediction (essentially about human behavior)--one that is clearly in dispute--but that can be resolved by observation. I suggest that Banana and GAG agree on how the "importance" of global warming should be measured relative to other issues, the criteria that must be satisfied for it to be the "burning issue of the next generation", and then in 30 years we can determine who is right.
6/02/2007 10:09 PM and C.C.,
You guys are right, formal theory results do not need any empirical component to be valid. the question posed was "Well, what is the utility of formal models over empirical methods?" So I was trying to show how they can be used as complements rather than substitutes.
But for the most part the kind of formal theory results you guys are talking about are in the social choice field. Most of the formal work in American (that board is where this discussion was born) are more in the vein of what I was talking about. Now I don't mean to say that any formal American work HAS to have an empirical component. Just that his work is more likely to be generating testable hypothesis than work in social choice.
On the other hand (and to possibly piss off a larger component of the board) I DO think that most empirical work would benefit from some formality whether it be directly in the paper itself or in reference to other formal work. Basing your empirical test on a comparative static or a computational result is a much more convincing test IMHO.
I think I love you, 6:06 am.
First C.C. said,
"...the variety of implementation problems that plagued the EPA - and arguably followed as a result of Congressional over-involvement in the policy area."
and then 9:50 pm said,
"I think the difficulty with your analysis ... is that it is excessively focused on Congress and the legislative aspects of environmental policy. An accurate understanding ... can only be fully grasped by also examining the implementation/administration of such policies. "
Huh? Your heart-felt beliefs are constricting your optic nerve.
My understanding of environmental policy "lives in" the nexus of Congressional, executive, and bureaucratic politics...both theoretically and empirically. So, respectfully, I am done with this debate.
Suppose that you believe theory is important but that you oppose the way that "formal" theory is currently practiced. For the sake of argument, suppose also that you generally oppose formalisation. What constitutes a good non-formal theory?
At the risk of irritating a small number of the board's readers, are analytical results superior to computational ones?
Perhaps you might want to tell us what you mean by formal? You seem to think computational theories are not formal, but some might disagree. I tend to think that if you write out a series of assumptions and then (explicitly) derive results from them, that qualifies as formal theory.
If you can show a result both analytically and computationally, I would rather have you show it analytically. In general, however, that's not the choice computational modelers face.
Anybody remember The Banana Splits? Kids show from the 1970s? Great stuff!
6/2 10:09 here.
I am mystified as to why anyone thinks formal theory with analytic results and computational models are substitutes or competitors, and the same for empirical methods like statistical modeling.
Even when they apply to the same type of question they are ways to touch different parts of the elephant. If they describe the elephant in the same way, it's a stronger indication that that's really the way it is than if any one of them describes it that way.
Or, you may have to learn to do or at least read more than one thing. Sorry to bear this bad news.
Hi Paul:
Your class looks like a challenge indeed. How have you changed the mix of readings and assignments over time? What subjects or readings tend to work very well? What have you dropped or added as you've taught it a few times? What is the biggest challenge to a successful class?
7:12 am here
My post posed two related but distinct questions. In the first, "non-formal" excludes both analytic and computational approaches.
The second was intentionally provocative, and it seems has achieved part of its purpose (making some people defensive). I see much value in computational models, and they are neither complete substitutes (and hence not direct competitors) nor perfect complements, but it would be useful to explicitly articulate the advantages and disadvantages of the two approaches.
Yes, I consider computational models to be "formal" because they satisfy 8:27's definition: "if you write out a series of assumptions and then (explicitly) derive results from them, that qualifies as formal theory."
However, in the analytical realm, "derive" means to "deduce" while in my (potentially incorrect) impression of computational practice, it means to "demonstrate" or "show."
To present the issue in a slightly different way, analytical theory seems to value general propositions over specific examples. One might liken computational analysis to involve harnessing the speed of the computer to do hundreds of thousands (or more) examples.
This is indeed useful, but should the computational modeler describe the general patterns observed and stop there? (For example, using the computations to produce "data" then running a regression.) Or should s/he seek to use the computational results to come up with general statements that can be proved analytically? (A practice that some formal (analytical) theorists employ but is not observed when the final research product is published.)
Formal models make deductive arguments but that's not generality. Whole formal literatures are based on what might properly be called examples, e.g. uniform distributions of some random variable and quadratic loss functions. The value in comparison to computational (esp. read as: agent based) modeling isn't generality, it's the deductive chain that shows explicitly which combinations of assumptions are responsible for which results.
Which, of course, is also why formal models don't uniformly get better as they get more 'realistic' but that's another topic.
I think that formalism is not something to be opposed, personally, but I understand the frustration that some feel when an article does not explain what is going on "behind" what appear to be notational shenanigans. But to be fair, this is analogous to those who do detailed case studies, non-formal theory, or quantitative work without explaining the corresponding method-specific jargon and providing motivations and "placement" within the braoder literature.
C.C. thinks we all agree that we prefer articles that we can read efficiently and fruitfully (HAH!) to those that we can't. :)
Regarding formal vs. computational, I agree with the sentiment that both are useful approaches, and that analytical results are, in the best of all worlds, preferable -- not because of added generality (it's best to hold the "generality of the result" constant, as slacker666 implicitly and wisely argues) -- but because -- again in the best possible world -- formal results can be demonstrated with less machinery.
For example, I can write code that demonstrates the law of large numbers, or I can prove (i.e., deduce) it analytically from first principles (i.e., independence).
The second approach, if done with care, is more direct and transparent, in my opinion.
Of course, I feel that computational methods are very useful. I always (at least attempt to) judge the appropriateness method by its ability to obtain and convey a finding or advance an argument.
All in all, C.C. is a glasnostian -- the choice of method, per se, is only relevant insofar as it is fruitfully applied to the topic at hand.
On the issue of lack of immutable social science laws:
Social science can never be certain that it has uncovered immutable laws of any kind. About this, the “Perestroika poster” is quite right. The counter argument, that social science can uncover probabilistic, but still immutable, laws is also wrong. For example, the p-values in a social science experiment tell us the probability that the experiment’s data patterns were generated by chance, given the underlying conditions of the experiment. If the experiment were repeated, we can never be certain that the underlying conditions are the same, or will be the same some time in the future. Perhaps genetic mutation will change the foundations of human behavior? Maybe global warming will make us all more ill-tempered? I would argue this fundamental uncertainty holds for any research method, large n or small (including case studies), and with any underlying theoretical foundation. Social science research can never be certain it has uncovered an immutable law of social science.
The Perestroika poster, and many arguing against him/her, are also wrong that physics, or any other “hard” science, can be certain that it has uncovered immutable laws of any kind. Just like with social sciences, the p-values in a physics experiment tell us the probability that the experiment’s data patterns were generated by chance, given the underlying conditions of the experiment. If the experiment were repeated, we can never be certain that the underlying conditions are the same, or will be the same some time in the future. (Please forgive the ignorant details in the examples I’m about to give; I’m just trying to illustrate a point. I know very little about physics.) In physics, perhaps the properties of the fundamental particles (muons and gluons and such) change over time so that even gravity will be different in the future? Perhaps in the future, the very nature of the dimensions we observe (and can’t yet observe, so the string theorists tell us) will change? Can we be sure that they won’t? Given the data we’ve observed so far, we think they haven’t or won’t change, but to be sure, we need to extrapolate beyond our data.
The Perestroika’s answer to the social science uncertainty problem is to dig deeper into the case studies to document the underlying conditions of the study. This is worthwhile, and in fact, all social science research should probably be more attentive to underlying conditions.
But I’ll end with the provocative statement: if there are laws in the physical or social realms, we’ll never know for sure if we have uncovered them.
For future reference, if anyone cares, you can call me “Spoon”
"But I’ll end with the provocative statement: if there are laws in the physical or social realms, we’ll never know for sure if we have uncovered them."
First, that statement is not really provocative.
Second, this debate is ages-old and very, very ready for a nap.
Third, your claim is itself a law, no?
Banana is succinct. I am not. Therefore, I will beat a dead horse instead of taking a nap.
Dear "Spoon",
This debate isn't really about "immutable laws". The obscurantist has strenuously made the following argument: in the physical ("hard") sciences, there exist "immutable laws" and therefore the associated methods (mathematical, experimental, statistical, etc) are appropriate. In the behavioral ("soft") sciences, immutable laws do not exist and therefore the methods of the hard sciences are not appropriate. (This is a paraphrase, of course.)
A key part of this argument involves the correspondence between "immutable laws" and "certainty". 11:14 rightly points out that this correspondence is false. It is perhaps a better explication of an argument that has already been sometimes too implicit on this board: that none of the approaches the obscurantist rails against depend on the existence of immutable laws.
One view of science is that the aim of science is not to discover "immutable laws" but to develop better and better theories or explanations for observable phenomena.
I do, however, take issue with what 11:14 suggests about certainty and our lack of it:
“we can never be certain that the underlying conditions are the same, or will be the same some time in the future.” The “PP” seems to take this statement and reaches a nihilistic conclusion: that we cannot establish empirical regularities and explanations of them (my working definition of “positive science”, however imperfect). I completely agree with the statement that “we can never be certain.” However, it does not follow that there are not empirical regularities that we can be reasonably certain will likely occur again. And yes, our uncertainty involves not completely knowing what initial conditions are.
This conclusion is reasonable enough: “Given the data we’ve observed so far, we think they haven’t or won’t change, but to be sure, we need to extrapolate beyond our data.”
However, I don’t understand this point: “The Perestroika’s answer to the social science uncertainty problem is to dig deeper into the case studies to document the underlying conditions of the study.” Yes, digging deeper can be enlightening and worthwhile, but while I’m willing to hear you out, I currently don’t see how, in the ideal case, a complete description of initial conditions resolves uncertainty.
-"Fork"
RE: 12:17 p.m.
The “PP” seems to take this statement and reaches a nihilistic conclusion: that we cannot establish empirical regularities and explanations of them (my working definition of “positive science”, however imperfect).
*****************************************************************
I do not see anything nihilistic in the propositions argued by any of the Perestroika interlocutors. To hold that the social sciences lack immutable laws to work with is not nihilistic. Indeed, the position at hand is not that knowledge is unattainable, but in the social sciences we should/must pursue the attainment of knowledge differently.
RE: 11:14 p.m.
Social science can never be certain that it has uncovered immutable laws of any kind.
********************************************
If we can detect immutable laws in the physical sciences, why could we not detect them in the social sciences if they existed?
Someone suffers from selective attention.
I would guess it's the guy saying the groups literature doesn't consider business interest representation.
RE: 12:17 p.m.
However, I don’t understand this point: “The Perestroika’s answer to the social science uncertainty problem is to dig deeper into the case studies to document the underlying conditions of the study.” Yes, digging deeper can be enlightening and worthwhile, but while I’m willing to hear you out, I currently don't see how, in the ideal case, a complete description of initial conditions resolves uncertainty.
*****************************************************************
Formal theory can help elucidate this discussion. Formal theory, for instance, points out how there are barriers to collective actions, questions of free riding, and certain difficulties with commonly held resources. Different people will respond differently to these matters. (This very reality is indicative of the fact that there no immutable laws of social/political behavior.) The fact that different people respond differently to the same quandaries means that to gain analytical knowledge requires case studies. We may be able to gain greater leverage on how/why different people respond differently (or sometimes similarly) to the same quandaries by conducting comparative case study analysis.
The fact that different people respond differently to the same quandaries means that to gain analytical knowledge requires case studies.
No it doesn't, not without additional assumptions you have declined to specify. The differences in response may be entirely idiosyncratic and random, or (equivalently) interpretable as random in light of existing theory about the behavior in question. That would be consistent with your stipulation, and case studies for that purpose are no more useful than a case study on the result of your last throw of a fair die.
RE: 6:43 p.m.
There is an irony here in that empirical thinkers literally have a faith in the existence of immutable laws of social/political behavior (i.e., universally recurring social/political behavior) where no evidence for such laws (universal behavior) exist. Stanley Fish posted a piece in the _New York Times_ today that is apropos:
"In a different, but structurally similar, scenario the failure of a predicted messiah to appear would be taken to mean (and has in history been taken to mean) that the faithful had been judged unworthy. If the belief is strong enough, if it is the cornerstone of one’s world-view, it will be no trick at all to re-characterize facts that others might see as a devastating challenge to it. (Thus in his 'On Christian Doctrine,' St. Augustine advises scripture readers who find parts of the Bible pointing a bad moral to work the text over until 'an interpretation conducing to the reign of charity is produced.') We don’t 'embrace' information that supports our beliefs; we see the information delivered by our beliefs."
"There is an irony here in that empirical thinkers literally have a faith in the existence of immutable laws of social/political behavior...where no evidence for such laws...exist."
Not actually ironic, though your highly dubious claim is. Irony would require that the faith in the existence (or the behaviors following from such faith) led to the nonexistence of such laws.
Mind you, the ironic part of this is that attempting to change minds about the proper means to pursue social scientific knowledge by arguing against the existence of "immutable laws" leads to active (and in C.C.'s opinion, justified) avoidance of the arguments concerning the proper pursuit of social scientific knowledge.
RE: 8:09 p.m.
C.C.,
The case study method has extensively been written about by a Perestroikan. Someone offered class analysis as a theoretical approach. No one has bothered to offer even mild criticisms of these, but yet they are seemingly rejected.
I don't get your immutable laws fetish and I am not sure I believe in them. I am also not sure whether I'm an "empirical thinker" but that sounds a bit grandiose for my taste.
I will say that statistical models do not assume immutable laws. They assume that behavior to be explained has an understandable part (i.e. can be rendered in terms of a theory of the behavior) and a not understood part. They assume that when the conditions of a decision identified as relevant by a theory are identical, the understandable part of behavior is identical.
Maybe that's what you consider an "immutable law" behind a statistical model. And maybe you think that's problematic because you are supposing two cases with identical conditions that are not identical in their understandable parts. (And why else would it be problematic?) But this does not refute or undermine the foundation of the model; it simply proposes an amendment to it. If you take two cases that one model treats as equivalent and say they should be different, you have simply proposed a different model. End of story.
Put differently, the (perhaps implicit) theories on which statistical models are based pose equivalence classes with respect to the behavior to be understood. Two members of the same class are understood to behave equivalently, at least as far as the theory can understand behavior. If you want to argue with the theory you say the equivalence classes partition the available cases incorrectly, e.g. some relevant variables have been left out.
And if you think about it, that's kind of what all positive (not to be confused with formal) theories in social science do; they assert equivalence classes of this nature. And that's what all theoretical debates are about. States and social revolutions? Same deal.
So I don't know if that's what this immutable law business is about. If that's what it refers to it's hardly specific to statistical modeling, and also hardly problematic. If that's not what it refers to it's irrelevant as a critique of statistical modeling.
It is a pity that this insipid discussion continues, but it has made me curious about something: What distinguishes the humanities from the social sciences?
I know which disciplines usually fall under the umbrella of the humanities (e.g. English, philosophy, history), but I don't know what its defining features are. What are its goals and objectives in terms of the pursuit of knowledge?
If it is possible to answer without using the term "science", then please do not use the term. Banana? Coffee drinker? Slacker666? Thanks.
"The case study method has extensively been written about by a Perestroikan. Someone offered class analysis as a theoretical approach. No one has bothered to offer even mild criticisms of these, but yet they are seemingly rejected. "
Huh? Is this a proposal of the non sequitur method?
Seriously, I'm not sure how I rejected anything in the 8:09 post except that the use of the word ironic. Oh, that and the presumption that the "immutable laws" discussion is useful or even relevant to the question of proper methodologies.
As posted earlier, C.C. has no problem with any method, per se.
Re: 8:51 pm
"What distinguishes the humanities from the social sciences?"
Good question. My first reaction was, "about $20,000 per year?"
My second reaction was, "C.C. thinks that the raison d'etre of the humanities is the synthesis of things that humans do and have done." The difference with social sciences is that the humanities are not interested in prediction. (C.C> thought about adding causation, but one difference is enough, and prediction seems to be the stronger case of the two.)
The analogy C.C. has in mind is that of a "literary scientist" whose work attempts to explain how the weather affects the choice of meter by poets or how personality traits affect the choice of short story or novel as a literary vehicle.
Just a shot in the dark, though.
"Huh? Is this a proposal of the non sequitur method?"
No, I think it is actually of the misdirection method (which uses the non sequitur method as one of its tools).
"The analogy C.C. has in mind is that of a "literary scientist" whose work attempts to explain how the weather affects the choice of meter by poets or how personality traits affect the choice of short story or novel as a literary vehicle."
Hmm...wouldn't this just be an unemployed psychologist? Although I think I get the point.
6/04/2007 8:51 PM
"What distinguishes the humanities from the social sciences?... What are its goals and objectives in terms of the pursuit of knowledge?
If it is possible to answer without using the term 'science', then please do not use the term."
------------------------------------
Uh...I don't know.
I have friends over in the humanities and I can't even get a foothold to begin understanding what they do or how they do it.
But in the spirit of pop culture references I have to say:
"Science!
Science again! I said science again!"
I would suggest that the key difference between the humanities and the social sciences is that in the social sciences the quest to identify causal mechanisms is paramount.
RE: 8:34 p.m.
Your explanation is not a defense, nor an explanation, of quantitative analysis. It could apply to virtually any empirical method. I believe it was John Stuart Mill that posited the statement that there is no "science without comparison."
slacker666 said, (following a nice discussion of the disconnect between "immutable laws" and the utility of quantitative empirical techniques)
"...If that's not what [the immutable laws `critique'] refers to, it's irrelevant as a critique of statistical modeling."
and then 6:50 am said
"Your explanation is not a defense, nor an explanation, of quantitative analysis. It could apply to virtually any empirical method."
So...6:50am agrees with slacker666. The immutable laws question is orthogonal to the utility of quantitative methods.
Whew, good. By CC's count, we're all on board, and we can finally let this immutably tired dog sleep.
RE: 7:03 a.m.
A key question is whether large-n studies or small-n studies serve as better bases to make comparisions and draw out causal mechanisms. As has been argued before, if there are immutable laws driving social/political behavior then large-n. If not, then small-n.
"As has been argued before, if there are immutable laws driving social/political behavior then large-n. If not, then small-n."
Not true. Regardless of what your view of social science dynamics is, more data is always better. Period. Note that this has nothing to do with immutable laws. See DeGroot, for example.
RE: 7:41 a.m.
Large-n studies can mean less data, not more. In order to work with many cases lots of pertainent information has to be omitted.
To be charitable to 7:11 (unlike that uncharitable banana!) there are tradeoffs. When I do large-n analysis I have to devote time to a bunch of different things, such as gathering data and rereading Greene, that reduce the amount of time I can devote to understanding particular cases.
That said, 7:11's claim that the choice between large and small n analysis hinges on one's belief in the existence of immutable laws is absurd, as is damn near everything that poster has written about immutable laws. Maybe being uncharitable with him/her is the right approach.
Only in political science could we find an argument that sample size determines prospects for causal inference.
The demands of causal inference are determined completely by one's definition of causation. If you are a counterfactualist, you need identical information in observations used for case studies as in data used for statistical models to identify causal effects, namely: counterfactuals. If the observation process allows you to observe counterfactuals either method will identify treatment effects; if not, neither will. Of course, as the banana points out, you want more observations than fewer. I am sorry to be hegemonic about this.
There is, for example, literally no difference in the ability of regression analysis and qualitative comparative analysis to identify causal effects in data.
Is it within the domain of social science to study "anonymous blog behavior"?
Let's see...should we use comparative case studies, class analysis, large-N quantitative analysis, or formal theory?
And by the way, though it's been observed many times, it bears repeating. Statistical modeling does not require "large N" per se. The Gauss Markov theorem does not reference sample size, for example.
"Pertainent"?
So not only do you offend statistics, but you also write poorly?
I thought you'd at least write nicely!
RE: 8:25 p.m.
I wrote the last post on the run. Sorry!
C.C. said
"Regardless of what your view of social science dynamics is, more data is always better. Period. "
and 7:48 am said
"Large-n studies can mean less data, not more. In order to work with many cases lots of pertainent information has to be omitted."
Ahhh... agreat example of an unfair rhetorical tactic. C.C.'s statement says more data is better than less. Again, full stop.
You are implying that large-n has less data than small-n.
In order to truly state whether large-n is better or worse than small-n, one would generally hold all things constant except for, oh I dunno, n?
In other words, for the final frickin' time -- if I have 200 observations of something (with level "X" of "pertinent information"), then I have NO LESS INFORMATION than if I had ANY subset of, say 100 of those 200 observations.
This is logically true -- regardless of whether you are doing case studies or regression analysis, as slacker666 rightly points out.
RE: 7:56 a.m.
When I do large-n analysis I have to devote time to a bunch of different things . . . that reduce the amount of time I can devote to understanding particular cases.
****************************************************
Then why do you have confidence in your findings?
"Then why do you have confidence in your findings?"
This is traced back to a shortcut known among quantitative scholars as the "Terval" (named after the Finnish statistician who invented the shortcut, which is much too complicated for a simple blog post). This approach is essential to understanding large-or-small-n analyses. Rarely described in detail due to their ubquity and complexity, you know a researcher is using this approach when he or she refers to his or her "confidence in Tervals."
confidence in Tervals
I heart C.C. Banana.
RE: 10:22 a.m.
C.C.,
In other words, for the final frickin' time -- if I have 200 observations of something (with level "X" of "pertinent information"), then I have NO LESS INFORMATION than if I had ANY subset of, say 100 of those 200 observations.
******************************************************************************
The more apt comparison is between a study with an n of 3, versus one of 100. With an n of three, all relevant variables can considered/incorporated and variable relationships can be carefully mapped. In a study of 100 cases numerous potentially pertinent variables have to be excluded, and variable relationships have to be greatly simplified. What is the analytical value of such a study?
The point is not about method A vs. method B, it's about the effect of more data on the quality of conclusions drawn from a fixed method. N = 100 beats N = 3 under any method because the latter situation can be rather well approximated by the former, if you ignore observations i > 3.
"The more apt comparison is between a study with an n of 3, versus one of 100. With an n of three, all relevant variables can considered/incorporated and variable relationships can be carefully mapped. In a study of 100 cases numerous potentially pertinent variables have to be excluded, and variable relationships have to be greatly simplified. What is the analytical value of such a study?"
This is so wrong, I don't even know where to begin.
Suppose you have 6 pertinent variables, each taking on 2 values (high/low). Then n=3 can not possibly allow you to "map" anything, carefully, or otherwise.
And, you're attempting again to stick your presumption that large-n => low-quality-n. I mean, hell, if you want to simply assume your desired conclusion, go ahead. But don't live under the delusion that you've proven something that is, in fact, logically false.
To do otherwise would be akin to me presuming not to have a peel. (HAHAHAHA. Get it? That's rich.)
RE: 10:57 a.m.
The real question is what have you proven with an n of 100? If your position is that there are no immutable laws driving political/social behavior, then all you have shown with a study of 100 n is that with the data you are working with "x" and "y" (frequently highly simplified versions of them) have a certain relationship. I have to question the analytical value of this finding.
The case study method does not necessarily presume that the findings produced have direct implications for other cases. Instead, a series of detailed case studies have to be pursued to gain insight into like cases.
I'll quote with one modification:
all you have shown with a study of n=? is that with the data you are working with "x" and "y" have a certain relationship.
This is actually a good description of "all" one has done with a case study method that "does not necessarily presume that the findings produced have direct implications for other cases," as you put it.
This is, for example, a good abstract description of the result of Skocpol's work on the conditions for social revolutions in France, Russia, and China, assuming there is no aspiration to speak beyond these cases. Under certain conditions in these cases, certain things happened. Under other conditions, other things happened. These are relational statements, in her case with the effect of one variable conditioned on the value of another (i.e., interactive in effect).
The puzzling thing is that you defend a method that can amount to this and no more than this when N=3, yet when N=100 you "have to question the analytical value of this finding."
RE: 11:20 a.m.
I will defend it to the extent that Skocpol's analysis could provide insight in those social revolutions. Additionally, her interpretation of these social revolutions could be applied to other social revolutions, and potentially give us insight into them.
In contrast, with a study of 100 cases, where numerous relevant variables are excluded and those variables considered are greatly simplified, what have we learned? I would submit that in most instances not much.
I think there is sometimes-to-often a fallacy in the way in which the assertion that case studies don't aspire to generalize beyond the case at hand is actually used. This is sometimes taken to mean that one does not seek to generalize beyond the -observations- in the data set at hand, but it does not mean that.
An important issue with any method is recognizing the importance of chance in creating the outcome that was observed. This is relevant even in case study research when one has no aspiration to speak beyond the cases at hand. It means that even in the fixed cases where particular outcomes obtained, those outcomes could have been different even under identical conditions.
A case study of social revolutions in France, China, and Russia may not attempt to speak beyond those three cases. But it does attempt to speak beyond the actual facts as they happened to unfold. It attempts to address the universe of possible results in those specific cases at hand, not only how things actually turned out but how they could have turned out. In implicitly assuming that chance plays a negligible role in determining the occurrence and outcome of a social revolution, it implies that the set of outcomes that could have occurred in those cases is the set of outcomes that actually did occur. Therefore, in identifying necessary and sufficient conditions for a social revolution in these cases one does not risk identifying factors that appear to matter empirically but really don't (which the statistically inclined recognize as a Type I error).
Even case study methods rely on a data generating process for the observations available. Interest only in the cases at hand is not the same as interest only in the observations in the data set. The cases could have turned out differently, if chance plays a role in social events. That being the case, analyses (case studies or otherwise) that assert no role for chance in determining outcomes, when in actuality chance does play a role, are seriously vulnerable to Type I errors. They are vulnerable to attributing the role of chance to some apparently systematic factor in the case.
her interpretation of these social revolutions could be applied to other social revolutions, and potentially give us insight into them.
This would seem to presume a systematic process by which conditions in a state map into likelihood of social revolutions in that state. Apparently you acknowledge that such a systematic process might exist, and therefore that generalization might be fruitful.
RE: 11:34 a.m.
"In implicitly assuming that chance plays a negligible role in determining the occurrence and outcome of a social revolution . . ."
*******************************************
I do not see the assumption in the case study method that "chance plays a negligible role in determining the occurrence and outcome." As a matter fact, researchers doing case studies will often invoke happenstance to explain particular outcomes. Of course, such happenstance takes place within a broader trend or set of structural factors.
I do not see the assumption in the case study method that "chance plays a negligible role in determining the occurrence and outcome."
That's because the assumption is not in the method per se, but rather in applications of it.
I think your problem in these discussions is that you are looking to identify or defend a particular method as the royal road to true knowledge, and critique another method as inherently and necessarily inferior. That is why you refract this conversation through the questions, "will the case study method force me to make errors?" and "will statistical methods prevent me from making errors?"
Banana, I think you have an identification problem. You misidentified the source of the important quantitative shortcut. You should have referred to T. Erval, who I think is either Brazilian or Portuguese.
RE: 11:36 a.m. and 11:53 a.m.
If "x" is important in "A1", could it be important in "A2"? Perhaps. Perhaps it is not of consequence in A2, but it is in A3, A8, and A12. The reasons for this could be beyond systematic analysis. People in cases A1, A3, A8, and A12 just choose to react a certain way to x, while in the other cases people just acted differently. This is the nature of knowledge in the social sciences. Again, people are cognizant and highly creative, and they respond in highly variable and unpredictable ways to stimuli.
Would I like our discipline to work like the physical sciences? Maybe. Am I looking for "the royal road to true knowledge, and critique another method as inherently and necessarily inferior"? I would say no. I believe I am arguing for methods that capture social science knowledge. Conversely, I am arguing against using methods because these methods mimic the physical sciences.
You are annoying us. You cannot make causal inferences from observational data, at least not easily. Fancier methods do not get you any closer to strong causal statements (and I know and use those fancier methods, incidentally).
Not a statement on either side, really. Just stop being sloppy with your language. Go read Judea Pearl.
Bayesians are not allowed here, either. :p
RE: 12:26 p.m.
You cannot make causal inferences from observational data, at least not easily.
***********************************************************
I would greatly appreciate it if you could expand upon this.
I would greatly appreciate it if you could expand upon this.
You have got to be kidding me.
See "Causal inference, fundamental problem of."
I believe I am arguing for methods that capture social science knowledge. Conversely, I am arguing against using methods because these methods mimic the physical sciences.
We agree that using a method because it "mimics physical science" is a bad idea. We appear to disagree about whether statistical models are used because of this mimicry or whether they require social science to be like physical science in some important (and not valid) way.
The correct answer is no to both questions. Statistical models require at least implicit theories that postulate relationships and they require a data generating process behind the observations in a data set. As you pointed out to me in a response above, and as is entirely correct, so does every other method that could be applied in positive social science.
RE: 12:39 p.m.
My point at this stage is that statistical methods in many instances do not capture sufficient knowledge, or captures it in a dubious manner (i.e., is knowledge actually being conveyed[?]). In contrast, we should have more confidence in the case study method.
"In contrast, we should have more confidence in the case study method."
"I would greatly appreciate it if you could expand upon this."
(said in a diabolical tone): MOOOHAAAAHAAAHAAA!
In a study of 100 cases numerous potentially pertinent variables have to be excluded
I suppose. I mean, we could call it "having a theory," but a more alarmist name would be "excluding potentially pertinent variables."
Also, I hereby curse this blog for disallowing html. What I was going to include was:
[large][large][large][blink]-->IMMUTABLE LAW<--[/blink][/large][/large][/large]
The ignorance displayed by the immutably moronic poster is really scary. In fact, I correct myself - his posts are clearly not of constantly low quality, but progressively revealing a degree of ignorance that I honestly find astounding.
No wonder Political Science has a bad rap.
Since ignorance is bliss, the dude(tte) must be happy.
RE: 1:34 p.m.
Certainly the more powerful critique offered here of quantitative methods is that variables and variable relationships have to be excessively simplified with the use of such methods.
"Certainly the more powerful critique offered here of quantitative methods is that variables and variable relationships have to be excessively simplified with the use of such methods."
Suppose that a relationship between variables, and the definitions of those variables, can be specified AT ALL. Then there exists a formal expression of this specification. And, accordingly, there exists a quantitative test of the specification.
Interestingly, the number of observations required to test the specified relationship is INCREASING in the complexity of the posited relationship and/or the number of variables one wants to include.
This is first semester quantitative methods.
C.C. is not trying to be rude, but seriously -- this is very well understood. And by understood, I mean formally and logically demonstrated.
Refer to the index entry, "Degrees of freedom." And then desist. :)
RE: 4:40
The assignment of numerical values to non-cardinal variables is a simplification of the data. In most cases, this simplification has a high analytical cost.
The assignment of numerical values to non-cardinal variables is a simplification of the data.
No it's not, not at all. If you have two possible outcomes --- social revolution or no social revolution --- there is no analytical cost to denoting one as outcome "13" and the other as "344." Or 0 and 1 for that matter.
The assignment of numbers simply reflects the substantive judgment that there are equivalance classes of outcomes of the social process. No cardinal properties of the numbers are used in this process.
RE: 5:01 p.m.
Let us stay "x" contributed to the Russian Revolution (assigned #1)and it did not contribute to the Chinese Revolution (assigned #2). Does x create a variance of .5?
RE: 5:08 p.m.
I would think the more important question is why "x" contributed to the Russian Revolution and not to the Chinese Revolution? Therefore, regardless of the specificity of the model and whether the numerical value in question really says something, I think the "knowledge" being transmitted here is shallow, or of little analytical use.
Let us stay "x" contributed to the Russian Revolution (assigned #1)and it did not contribute to the Chinese Revolution (assigned #2). Does x create a variance of .5?
Your question is incoherent because "contribution" is not observed. It is theorized and possibly inferred from relationships present across observations.
To make your question coherent let's say x = 1 in two cases, one of which experiences a social revolution (y = 1) and one not (y = 0). Then, conditional on x = 1, the variance in the dependent variable for the universe of those two cases is .25 = P(y = 1|x = 1)[1 - P(y = 1|x = 1)].
I would think the more important question is why "x" contributed to the Russian Revolution and not to the Chinese Revolution?
Well, there's the question of whether and then the question of why. A statistical model can never answer the latter but the former is pretty important too. And a statistical model can give you a piece of information about that.
RE: 6:03 p.m.
The point I was trying to communicate is that "x" is present in both cases, but it is only in one of two cases that "x" actually contributed to revolution. So let us say both countries had an equal level of industrializtion, and in one case (Russia) industrial workers lead the revolution, but in the other (Chinese) industrial workers did not have any role in bringing about the revolution. Instead, it was brought about by peasants. How would numerical analysis lend insight in this instance?
RE: 6:12 pm
This is covered in an introductory research design course and the resolution is the same for any methodology, quantitative or qualitative. The assumption of "cardinality" is irrelevant.
Again, C.C. does not mean to be rude, but these questions (whether from one or many source with similar use of "RE:") reveal a fundamental misunderstanding/lack of training in inferential methods (again, qualitative or quantitative).
RE: 6:26 p.m.
Then the answer should be simple.
RE: 6:04 p.m.
Well, there's the question of whether and then the question of why. A statistical model can never answer the latter but the former is pretty important too.
****************************************
The case study method allows the researcher to get at why.
"Again, C.C. does not mean to be rude, but these questions (whether from one or many source with similar use of "RE:") reveal a fundamental misunderstanding/lack of training in inferential methods (again, qualitative or quantitative)."
well, it is kind of rude to suggest that someone has been poorly trained...
why do you insist on responding to someone who would clearly fail a qualifying exam (or even an undergrad midterm) in research design? me thinks s/he would fail logic and rhetoric as well, which significantly predate modern science and statistics.
i'm sure s/he is very good at class analysis and comparative case studies, but it is painfully obvious (i am 99.7% confident of this) that in this case(!) such expertise does not carry over to justifying his/her preferred paradigm or undermining the dominant paradigm (broady conceived).
let's move on to less settled (at least on this blog), but no less controversial topics...
what's the deal with the bayesian statisticians? should we follow their lead and abandon our silly frequentist approach (because of its inferior philosophical basis)?
RE: 6:04 p.m.
Well, there's the question of whether and then the question of why. A statistical model can never answer the latter but the former is pretty important too.
*******************************************************
Let us be very clear. It has already been explained that quantitative analysis in the social sciences is not predicated on the existence of laws of social/political behavior. Therefore, insofar as the word "whether" implies the future, statistical models cannot tell us whether. What they can tell us is that a statistical relationship exists presently or in the past. Scientifically speaking, this does not imply that the relationship will exist in the future.
RE: 6:54 p.m.
reveal a fundamental misunderstanding/lack of training in inferential methods
************************************************
Without immutable laws of social/political behavior underlying quantitative models, the inferences drawn from them are weak, especially when compared to the physical sciences. If the inferences are equally weak in the case study method, I would like to hear why.
Christ is this a lame discussion. Two groups of immutable dunderheads! Give it a rest already! Do you think you're going to persuade the other side?
RE: 7:23 p.m.
I have found this discussion helpful insofar as it has clarified my thoughts on this matter.
"Do you think you're going to persuade the other side?"
No, I doubt one side believes this. Maybe both sides don't. My theory is that this behavior is driven more by ego, a preference for self-expression, and procrastination (from doing real work in which we put our arguments in print with attribution). But maybe I am projecting my own motives onto others.
I also think these knuckleheads have developed a great deal of useful teaching material (along with the chaff).
I also think these knuckleheads have developed a great deal of useful teaching material (along with the chaff).
***********************************************
While I don't appreciate the "knucklehead" barb (lol), I am glad you have found the discussion useful too. =-)
"Christ is this a lame discussion. Two groups of immutable dunderheads! Give it a rest already! Do you think you're going to persuade the other side? "
Only if there are no immutable laws in the science of social science.
Yeah! C.C. is a dunderhead!
Not to be confused with C.C.'s AC/DC cover song, "dunderstruck."
This comment has been removed by the author.
"I also think these knuckleheads have developed a great deal of useful teaching material."
Yeah! C.C. is a knucklehead!
As a result of this discussion, C.C. *has* developed a very sophisticated screening method for graduate admissions to his/her university.
Look, love it or leave it. Or meet C.C. in the bar at APSA. Two fingers of Bookers, on him/her.
Man, you don't sign in for a few days and you miss 100 comments on Popper ...
===
Paul:
What readings/articles go over especially well (or not so well) in your class? How interested do the students get? I would assume that this stuff isn't what they imagined when they signed up for a Political Science class.
===
Two things to respond to there. First, on what they expect from a Pol class. I am at a particular sort of institution, where the stated ambition of a significant proportion of the students is to go on to PhD programs. So the sort of "civics class" expectations aren't a problem here. And I have the benefit of tenure, so if students aren't that happy, they won't take my upper division classes. That's OK.
More of an issue is when I put equations on the board the first day. And we have a year long required Humanities sequence, so the first style of analysis that students learn is humanistic, and curing them of ... ha ha ... showing them alternatives to that is a challenge.
In terms of readings, the ones that remain on the syllabus are the ones that work (for me).
The Plous is really good.
The Hardin is tough sledding.
I absolutely love the Jenkins and the Jenkins and Sala. The combination of a well defined problem + history + good discussion of the methods. (Never quite sure that Jeff likes it when I email him every other year saying how well they teach.)
Paolino et al. doesn't work so good. Classic Page and Brody works well.
Interestingly, in many of my classes, I've found the various work by Poole and Rosenthal works really nicely. The website is a huge plus.
Not to be confused with C.C.'s AC/DC cover song, "dunderstruck."
That wasn't AC/DC. That was originally by V.I. "Dirty Deeds" Dundersheep.
"A key question is whether large-n studies or small-n studies serve as better bases to make comparisions and draw out causal mechanisms."
Color me confused, but what is a causal mechanism? The term gets trotted out in regular defense of the case study method (which I support and use), but I fail to understand just what a causal mechanism is, if it's not either (i) a series of relationships between intervening variables, or (ii) simply a social scientific-y dodge which says little more than "some stuff happened."
Anyone? Anyone?
RE: 5:01 p.m.
Let us stay "x" contributed to the Russian Revolution (assigned #1)and it did not contribute to the Chinese Revolution (assigned #2). Does x create a variance of .5?
6/05/2007 5:08 PM
-----------------------------------
Sweet Jebus,
Have you taken a single quantitative methods class. This is like day 4.
RE: 11:55 a.m.
Color me confused, but what is a causal mechanism? The term gets trotted out in regular defense of the case study method (which I support and use), but I fail to understand just what a causal mechanism is, if it's not either (i) a series of relationships between intervening variables, or (ii) simply a social scientific-y dodge which says little more than "some stuff happened."
******************************************************************************
What statistical methods can do is establish correlation. In other words, when "x" is present "y" occurs in 50 percent of the cases. If this were the physical sciences, we could deduce why this relationship exists. As acknowledged, there are no immutable laws driving social/political behavior, so all we are left with in the social sciences is a statistical correlation of 50 percent between x and y. Thus, scientifically speaking, in the social sciences having correlation does not imply causality.
With the case study method, the why of this correlation be can addressed. In other words, a researcher can get at why the Chinese peasants revolted, but not the Chinese industrial workers. Conversely, they can provide an answer as to why the Russian industrial workers revolted, but not the peasantry. The mind set of these different groups can be explored, as can their motivations. There will be debate and disagreement over conclusions and findings, but the researcher using the case study method is not limited to solely establishing correlation.
I respectfully submit that you have no idea what you are talking about.
Doesn't matter how hard you look at the “motivations” of anyone, if you can't independently control the levels of your IVs (or explanations, if you prefer) all you doing is establishing correlations, whether your N is 1, 2, or 2000. Any criticisms you raise on large-N methods on this score applies to your preferred design as well.
I suggest you look at a research methods syllabus. Anyone with serious training – quant or qual – could set you straight.
As acknowledged, there are no immutable laws driving social/political behavior
-----------------------------------
First of all we have acknowledged you have said it. I am not sure if anyone has acknowledged it is true.
What has been said is that it may or may not be true that there are immutable laws. But that it does not harm quantitative analysis if they do not exist.
How many scholars both qant and qual have to tell you that your line of reasoning is all washed up for you to have some doubt in what you are saying.
RE: 8:56 p.m.
What has been said is that it may or may not be true that there are immutable laws. But that it does not harm quantitative analysis if they do not exist.
********************************************
It is the immutable laws of physics that give the physical sciences their analytical power, and, more specifically, their quantitative methods analytical leverage. These laws is what allows physical scientists to convert their statistical correlations into causal explanations. Again, in the absence of immutable laws, social science quantitative correlations remain that -- correlations.
RE: 8:11 a.m.
Doesn't matter how hard you look at the “motivations” of anyone, if you can't independently control the levels of your IVs (or explanations, if you prefer) all you doing is establishing correlations, whether your N is 1, 2, or 2000. Any criticisms you raise on large-N methods on this score applies to your preferred design as well.
****************************************
So what you are saying is that if someone says they shot their spouse because he/she was having an affair, we cannot conclude that the affair was causal in the shooting of the spouse?
RE: 10:27 a.m.
These laws is what allows physical scientists to convert their statistical correlations into causal explanations.
**************************************
Can you give an example of how this works in the physical sciences?
11:55 pm here again.
"With the case study method, the why of this correlation be can addressed. In other words, a researcher can get at why the Chinese peasants revolted, but not the Chinese industrial workers. Conversely, they can provide an answer as to why the Russian industrial workers revolted, but not the peasantry. The mind set of these different groups can be explored, as can their motivations. There will be debate and disagreement over conclusions and findings, but the researcher using the case study method is not limited to solely establishing correlation."
You certainly didn't come close to answering my question, that's for sure. I understand statistical inference quite well, and I understand the idea of closely looking at a case. I just don't get what a 'causal mechanism' you speak so highly of is. You speak of the motivation of the peasants, but beyond that being impossible to establish (those in the black earth region certainly had different interests than those in central European Russia, for example - a result of things we can code, like patterns of land holding etc.), I don't see how that shows causation any more than it shows correlation.
I certainly don't see how looking at individual motives (let alone "group" motives) provides us with a mechanism to show causation. This isn't a thinly veiled quantoid attack on qualitative methods. This is a sincere question as to just what people mean when they hand-wave over such terms, and if we can even come to a consensus as to what a "causal mechanism" might be, beyond either a variable or "stuff happened."
I just noticed that you can collapse individual comments (so that you don't see the comment). Very handy.
RE: 10:58 a.m. and 11:24 a.m.
The amount of heat in water directly correlates to the state of water (i.e., gaseous, liquid, or solid). In the physical sciences because there are immutable laws at work such a correlation will be assumed to be causal – heat causes water to change states.
In the social sciences there are no such laws in operation. Thus, you do not generally, if ever, find the kinds of correlations involving social behavior that is found in the physical sciences. Secondly, when correlations are found involving social behavior, because there are no laws driving social behavior, it is scientifically appropriate to assume that such correlations are not causal. "Correlation without causation" is a common refrain in the social sciences.
In the social sciences, however, our subjects can state their reasons for their behavior. In other words, an atom of oxygen cannot tell us why it bonded with carbon, but a human can tell us why he/she participated in a rebel army. Now, is consciousness decision-making the only factor in behavior? No. Nonetheless, understanding the rebel army's ideology (speeches, policy documents, interviewing, soldiers, political leaders, and military officers) can allow the researcher to gain analytical insight into the specific reasons causing a rebellion.
Psychologists tell us that individuals do a terrible job of describing their own decision processes. Self-reports of the decisions leading to a behavior may be orthogonal to the real reasons. See Timothy Wilson’s work for example.
ttp://people.virginia.edu/~tdw/
RE: 12:34 p.m.
I acknowledged that conscious decision-making is not the only factor in determining behavior. Nonetheless, to ignore people's stated reasons for their behavior is seemingly anti-positivist.
RE: 12:34 p.m.
People collectively express political opinions and desires that are surprisingly rational given their self-interest. The poor, for instance, will collectively want income redistribution for the poor, and the rich will collectively oppose such a redistribution. This leads public opinion researchers to conclude that there is a collective political rationality.
From this it can logically follow that gauging the collective's opinion on their motivation(s) for a certain political action (e.g., revolt) you can be gauging the collective's actual motivation(s). Put differently, people, as a whole, can effectively communicate to the researcher (through various means) the actual causes of their actions.
Dear ILP (Immutable Law Poster),
For the sake of argument I will grant you that there are no immutable laws in social science. So lets take that discussion off the table for a moment.
Please explain exactly where the lack of immutable laws causes quantiative analysis to run awry.
Simply stating that the lack of such laws weaken quantiative methods is not enough for us. Why?
*Explain exactly where quantitative methodology (formal or empirical) relies on immutable laws.*
I for one will not listen to another word you say until you explain this. You have said it over and over but provided no mechanism by which the lack of immutable laws would do violence to most quantitative work.
ILP would argue (I am not ILP) that the lack of immutable laws means that any conclusions we draw from our empirical analysis today might not hold tomorrow if the underlying conditions change.
However, s/he has not explained why this is a particular fault of quantitative or empirical methods; it would seem to me that the case approach would even be more vulnerable to the inability to generalize when underlying conditions change. E.g. there's zero Russian Revolutions between 1200 and 1917, then there's two in one year, both of which make fundamental alterations in the political order. I'm not sure what understanding the first 700 years buys you in understanding 1917, except that everybody who wasn't a Romonov didn't like the status quo all that much and an external stimulus might have triggered a coup or some other instability.
*yawn*
RE: 2:02 p.m.
It is important to stress that we gain knowledge of the laws of physics through the behavior of matter. Thus, we cannot see or touch gravity, but we know it exists because of the behavior of matter. Thus, as scientists study the movement of matter they are working with/gauging the laws of gravity.
Now, many of the laws of physics, or physical processes, are detected/measured through statistical correlation. We "know" smoking causes lung cancer, for the most part, through statistical analysis. We may not fully understand the physics of the relationship between smoking and lung cancer, but the statistical correlation between the two allows us to assume that there is a physics (or physical process) there linking them in causal manner.
In the social sciences there is no evidence of laws driving/determining social behavior. As a result, statistical correlation found with social behavior is scientifically assumed to be coincidence. Now, are all statistical correlations in the social sciences the result of coincidence? No. But unlike the physical sciences, a strong statistical correlation between variables cannot be assumed as evidence of causality.
It is worth noting that in the social sciences statistical correlations tend to be much, much lower than found in the physical sciences. This in-and-of-itself is taken as evidence of the lack of "laws" driving/determining social behavior. When this is added to the fact that social scientists cannot reliably model the future of human behavior (also the result of the lack of social/political laws), an opinion one hears from the physical sciences is that our discipline is not a science at all. An opinion, I will quickly add, that I do not share.
RE: 2:11 p.m.
We cannot narrowly predict human behavior. Instead, the objective of social science is retrospective analysis (i.e., understanding what has happened in the past). Does this give us a guide to the future? Perhaps, perhaps not.
I believe the case study method is an excellent tool for retrospective analysis. Often it is superior to all others.
Lung cancer is an interesting example given that 1) your preferred case study method will have a hard time reliably detecting it give that only about 1 in 3 smokers gets lung cancer, and 2) Some people get lung cancer who do not smoke (and are not exposed to second hand smoke).
So much for the laws of the physical sciences too I guess ;-)
ER: 2:43 p.m.
What you are arguing is that laws mediate the relationship between matter, and statistical correlation is one of the ways these laws are measured and detected? In contrast, laws do not govern or mediate social behavior, and, as a result, we are uncertain what statistical correlation is capturing with regard to social behavior?
For those interested in formal papers for undergrads, here's two I have used:
"Farquharson and Fenno: Sophisticated Voting and Home Style." Arthur Denzau, William Riker, Kenneth Shepsle. The American Political Science Review, Vol. 79, No. 4 (Dec., 1985), pp. 1117-1134.
"Regulatory Regimes, Agency Actions, and the Conditional Nature of Congressional Influence." Charles R. Shipan. American Political Science Review (2004), 98: 467-480.
The first is a nice (Though disputed if you read subsequent literature) example of sophisticated voting and agendas, and the latter is a nice example of the spatial model.
Does anyone wonder how much giggling the ILP does when s/he sees us twist ourselves up trying to respond? It could just be a big joke, people.
RE: 2:43 p.m.
You are truly completely ignorant of quantitative methods. There are tests to determine whether the variance measured by a model is the result of happenstance or not.
RE: 7:37 p.m.
I am aware of these tests. The problem is sometimes social behavior very much appears linked to a cause, but it simply is not. Its like running into your best friend in a far away city. On the face of it, such a meeting appears as planned, but in actuality its happenstance. So the skeptic can legitimately hold that the data looks like it properly correlates, and it survives statistical tests, but in the absence of laws of behavior being demonstrated the correlation may still be happenstance.
I have a fairly standard set of basic quantitative/formal tools: statistics and econometrics (i.e. most of greene), game theory and some social choice. Enough so that I can work within the current quantitative/formal paradigm in American politics.
There are many advanced methods that I am aware of, have a vague idea how they can be used, but do not have enough familiarity with or time invested to be able to use them. What kinds of new or advanced methods are worth investing in, other than comparative case analysis? Computational modeling (so-called ABM), evolutionary GT, social network analysis, Bayesian statistics? Are they fads? Should more people use these tools, or only a select few?
ILP,
Lets try again. You said:
"In the social sciences there is no evidence of laws driving/determining social behavior. As a result, statistical correlation found with social behavior is scientifically assumed to be coincidence."
Why? Explain to us why this is true? What part of the lack of immutable laws causes us to have to assume that any correlations are coincidence?
RE: 5:44 a.m.
Why? Explain to us why this is true? What part of the lack of immutable laws causes us to have to assume that any correlations are coincidence?
*****************************************************************
The relationship between matter is mediated by immutable laws (gravity perhaps being the most important). Therefore, when variable correlation is found within/between matter it is assumed that the laws of physics are underlying (driving) the correlation. No such laws mediate/shape human behavior, thus in a definite scientific sense correlation involving human behavior is assumed to be happenstance/coincidence.
Think of it this way, there are invisible linkages (laws of physics) extant between all forms of matter. As matter behaves, it is assumed that these linkages are determining (shaping) this behavior. For humans, there are no such linkages, thus human behavior from a certain scientific perspective is viewed as totally random.
Obviously, there are patterns of human behavior, but these patterns are not determined by "linkages" but by human cognizance and creativity. This fact makes the patterns of human behavior difficult or impossible to predict, and it also greatly weakens our ability to rely on statistical correlation to analyze the causes underlying these patterns.
What does ILP mean?
Anonymous said...
What does ILP mean?
6/07/2007 6:28 AM
------------------------------
Go to the top of the page.
Hit CTRL+F
Type "ILP"
This will show you the first use of "ILP".
"You can learn all these skills and more just by calling 1-800-555-5555. I will send you my product 'How to use a web browser' free of charge. All you have to do is pay the small postage charge. I know once you try my product you will be back for more...Try My Product!"
coffee drinker: even though I am a fork, hopefully I can still be of some use to you.
don't try to understand the ILP's tortured logic. S/he keeps coming back to this "immutable law" premise, which reveals quite a bit of ignorance about modern philosophy of science:
"The relationship between matter is mediated by immutable laws"
As has been noted many, many times on this board, this in and of itself comes quite close to be the only "immutable law" of social behavior.
RE: 7:43 a.m.
don't try to understand the ILP's tortured logic. S/he keeps coming back to this "immutable law" premise, which reveals quite a bit of ignorance about modern philosophy of science
******************************************************************************
The "laws of physics" serves as the basis of understanding of all physical sciences. To suggest otherwise is beyond belief!
7:43 am
To negate the laws of physical reality is to lapse from a profound lack of understanding of science into a hostility to it. Even Christian fundamentalists do not deny the laws of matter. Instead, they take them as a sign of God's existence.
Did anyone read the set of stories in the Chronicle of Higher Education recently on the US News Rankings? Any thoughts? One thing that really stood out was the difficulty public universities have in competing, based on the ranking criteria.
I have two close friends that work in the physical sciences (a physicist and biologist), and they convey much of the criticisms laid out here of the social sciences. Quite frankly, they are more derisive. At first, like many of you, I did not fully understand their position, and, perhaps, I was off-put by their dismissive tone. As I explored the subject with them, and they showed more patience with me, I came to see what they were arguing. It greatly deflated my confidence in my research, and even in my professors. But I still believe that my work and quant methods are important and "communicate knowledge" – as it has been put.
You people are really good at baiting us into this silly morass.
To go back to the history of the science, at one point, people believed that the earth was stationary, did not spin on "its" axis, and that the sun, stars and planets revolved around the earth. Prior to (and even for some time after) Copernicus, these were the so-called "immutable laws" of the physical world.
Do you believe otherwise? Why?
Of course, a "scientist" doesn't actually need to know--or defend--whether there is such a thing as an "immutable law" or not. They work within their scientific paradigm, make advances in knowledge, and as far as they know, there are "immutable laws" of physics, even if they don't truly exist.
Since someone brings up Christianity, let me open up the possibility for ad hominem and other rhetorically useful but otherwise logically flawed arguments. I regard Christian fundamentalists and others who require proof of the existence of God to have a weak faith. My own beliefs border on the "fundamentalist" with regard to the "truth" of Scripture and its revelation to us of the existence and nature of God (in three persons). Do I need absolute, physical, "scientific" proof? No, I have faith: "being sure of what we hope for and certain of what we do not see."
I've never had much luck treating physical scientists as authorities on social science, substance or methodology. They tend not to know what they are talking about, and have partial knowledge of the field from dimly remembered undergrad classes.
We don't and won't achieve the predictive success of physical science because we face harder problems. Murray Gell-Mann, a nobel prize winning physicist, made this point many times about social science.
I got to know a few natural scientists in grad school, and they were in top departments in their fields. They were pretty dismissive of social science methods, without doubt, but also pretty ignorant. It was more like a presumption of superiority than a result of any reflection. They didn't have much idea of what social science does or how it tries to do it...attempts at substantive debate usually ended with "well you can't do experiments and your data has too much randomness" etc. So what are we supposed to do, leave the questions to witch doctors and astrologers?
That said, if you let the natural scientists take an authoritative posture I have no doubt they would dismiss everything we do and act like they had the best principled reasons for it.
>> Since someone brings up Christianity, let me open up the possibility for ad hominem and other rhetorically useful but otherwise logically flawed arguments.
Don't worry, you could make a bunch of highly-inflammatory remarks about religion (pro or against) and you wouldn't generate a reaction here as strong as what happens when people start debating methodology.
RE: 8:35 a.m.
We don't and won't achieve the predictive success of physical science because we face harder problems.
******************************************************************************
So your position is that what political scientists do is "harder" than brain surgery, space flight, the construction of skyscrapers, or the building and running of nuclear reactors? I have to ask, what is your definition of "harder"?
Funny how none of those are actually science, but surgery or engineering.
RE: 8:32 a.m.
Of course, a "scientist" doesn't actually need to know--or defend--whether there is such a thing as an "immutable law" or not. They work within their scientific paradigm, make advances in knowledge, and as far as they know, there are "immutable laws" of physics, even if they don't truly exist.
******************************************************************************
The evidence for the immutable laws of physics is found in the ability to predict the behavior of matter. Unless you have opted out of the modern economy, then we all believe in the immutable laws of physics, since medicine, electronics, air transport, building construction, etc. are all based on humanity's understanding of the immutable laws of physics.
RE: 10:49 a.m.
The Perestroikans were cast as Luddites, but as it turns out it is the proponents of quantitative methods that are the real Luddites. They feel they have to pull down science, and the technology it is based upon, a notch or two in order to justify their work.
Of course, a "scientist" doesn't actually need to know--or defend--whether there is such a thing as an "immutable law" or not. They work within their scientific paradigm, make advances in knowledge, and as far as they know, there are "immutable laws" of physics, even if they don't truly exist.
******************************************************************************
Perhaps turning the discussion in a more constructive direction, if the physical sciences are based on the assumption of immutable laws of physics, what assumptions are political science predicated upon?
RE: 8:35 a.m.
We don't and won't achieve the predictive success of physical science because we face harder problems.
******************************************************************************
You are absolutely correct. The work of the political science is harder because our subject is cognizant and highly creative, and as a result capable of behaving in completely random ways. This makes narrow prediction near impossible, and statistical inference is always in question.
ILP said:
The relationship between matter is mediated by immutable laws (gravity perhaps being the most important). Therefore, when variable correlation is found within/between matter it is assumed that the laws of physics are underlying (driving) the correlation. No such laws mediate/shape human behavior, thus in a definite scientific sense correlation involving human behavior is assumed to be happenstance/coincidence.
Think of it this way, there are invisible linkages (laws of physics) extant between all forms of matter. As matter behaves, it is assumed that these linkages are determining (shaping) this behavior. For humans, there are no such linkages, thus human behavior from a certain scientific perspective is viewed as totally random.
----------------------------------
Is the crux of your argument that when a hard scientist finds a correlation they are able to ascribe causation based on the fact that there are immutable laws of physics? And we in the social sciences have no immutable laws and therefore when we find correlation we are unable to ascribe causation?
What do we think about making the following distinction:
The natural sciences have laws that exist independently of whether or not humans want them to exist. Gravity will exert the same influence on matter regardless of where you are in time or place.
There are general laws in the social sciences, for example, we know with relative certainty that most African Americans will vote Democrat in the next presidential election (even though our predictions regarding the exact percentage are inexact). We also know things like poor economic performance hurts incumbents. These things may or may not be “important,” an issue that involves the value-neutrality of social science research, but you cannot deny that there are regularities in politics that we understand and can predict with a fair degree of certainty (although with perhaps less certainty than the natural sciences)
However, the laws (or “law-like” regularity) that exist in a given society do not hold true independent of time and place. At an earlier period in American history, African Americans favored the Republicans, not the Democrats.
This indicates that there is a historical dimension to the social sciences that does not exist in the natural sciences (with perhaps the exception of biology). It also indicates that in addition to figuring out what the general “laws” are in contemporary societies, we must try to figure out why these laws, and not others, exist.
Sincerely,
A Historically Oriented Social Scientist
RE: 11:36 a.m.
Those are key aspects of the argument being put forward.
Gravity will exert the same influence on matter regardless of where you are in time or place.
In fact, it won't. Even the measured effects of gravity will differ depending on the depth and steepness of the gravity well you're in. And my own presence or absence will affect the gradient of local spacetime deformation... just not very much.
There are general laws in the social sciences, for example, we know with relative certainty that most African Americans will vote Democrat in the next presidential election
That's such a completely stupid example of a "law" that I have to think you're just building a straw man to knock down. No reasonable person would offer that as a "law" because there's no theory behind it. Better, but still firmly on Planet Dumbass, examples drawn from that same observation might include "Affiliation with a political party is shaped by both early family history and by political experiences, particularly in young adulthood," or "People disproportionately tend to vote in what they perceive as their economic interests, subject to sharp informational constraints."
Your (or whoever's) preoccupation with this law nonsense is tiresome. There aren't immutable laws governing political life. Okay. So? Fine, we can't do the equivalent of sending man to the political moon, at least not until we can model political thought and behavior through direct, full-scale simulations of human mentation as a system of physical processes.
We can still do a damn sight better than just guessing. And we can do a damn sight better than guessing combined with an appeal to the authority of the guesser. We can do better than guessing by coming up with theories, beating all of the empirical implications that we can out of them, and using our predictive failures to refine our theories. That is, by doing science, we can learn more than otherwise.
So your position is that what political scientists do is "harder" than brain surgery, space flight, the construction of skyscrapers, or the building and running of nuclear reactors?
Of course it's harder. All that they're doing is shoving dumb matter around. We're not just trying to understand aspects of human mentalities one at a time, we're trying to understand aspects of how human mentalities interact in a common environment. Compared to that, stellar physics is dead easy.
I agree with your conclusions, even if I do not agree with your tone. A key question that your otherwise informative post did not address, however, is which method(s) make(s) us as a discipline "better guessers"?
"The evidence for the immutable laws of physics is found in the ability to predict the behavior of matter. Unless you have opted out of the modern economy, then we all believe in the immutable laws of physics, since medicine, electronics, air transport, building construction, etc. are all based on humanity's understanding of the immutable laws of physics."
Ah, despite the sly argumentum ad hominem and (not so sly) ad populum embedded in this argument, this brings us back to prediction, doesn't it?
This discussion has had two related, but distinct components, and continues to devolve for the worse. One is epistemological: what can we know from our methods? The other is ontological: do there exist "immutable laws" or not?
The first is of greater concern to us, and what this discussion was originally about. The second is only of concern insofar as the correct deductions and inferences we make from our methods depend on the existence of "immutable laws".
The ILP claims that they are, but still has not produced a compelling argument why we should believe this to be the case: s/he only continues to assert the necessity of "immutable laws" rather than proving it from first principles.
Would the ILP stipulate that there may be "law-like regularities" (thanks HOSS--can we call you that?-- for bringing this term up), which serve the same purpose in the social sciences as the so-called "immutable laws" in the physical sciences.
The argument I am making about doubting the existence of "immutable laws" of the physical world is that the same arguments the ILP uses against quantitative methods in the social sciences apply also to the physical sciences. How do you know with certainty that the universe will not collapse tomorrow? You don't with absolute certainty. This point has already been made. Whether or not there are truly physical laws does not matter if we have theories that predict well and so allow us to make technological advances. Although I don't necessarily agree that prediction is everything, this is certainly an important component of scientific knowledge.
The relevant question is how do you know that? Is it by faith alone? (So far, that is what the argument seems to be.) Is it because the scientists tell us what these physical laws are? Or is it inductive--the universe has not collapsed in the past and so we are sure it will also not collapse in the future?
"Perhaps turning the discussion in a more constructive direction, if the physical sciences are based on the assumption of immutable laws of physics, what assumptions are political science predicated upon?"
Perhaps at a fundamental level, we can safely assume that people have wants and needs. This can certainly be historically, culturally, or whatever contingent. That is a long way, however, from allowing us to make any kind of predictions with reasonable certainty. We need to know what those wants and needs are (maybe we'll call these preferences), but we'll also want to know how people pursue them--what are their decision-making processes?
But perhaps the point of the scientific side of social science isn't to discover immutable, universal laws. It should be to gain an understanding of "(law-like) regularities" as well as the context and conditions under which they occur--which themselves may be law-like statements. Rather than trying to find that "Y" occurs, we want to find regularities of the form "if X then Y". The initial conditions X might fleeting. They might persist for decades or centuries. But then we can push the question back so that we can ask: "why did condition X occur when it did"? And so the pursuit of knowledge continues.
This is a far cry from arguing that "because there are no immutable laws of social behavior, we cannot use quantitative methods because the comparative case study method is superior."
Yes, there may not be "immutable laws" of social behavior. I'll stipulate that there aren't, but that's entirely irrelevant to understanding how we know what we do and our certainty in what we know.
Wow...I have wasted a lot of time today. Chalk one up for irrationality.
Good grief, don't you folks have articles/books to write, papers to grade, even naps to take...It's exhausting just scanning, certainly without reading, any of this...but hey, to each his own....
RE: 1:47 p.m.
This is a far cry from arguing that "because there are no immutable laws of social behavior, we cannot use quantitative methods because the comparative case study method is superior."
*************************************************
That has not been asserted on this blog nor the earlier one.
If it is acknowledged that there are no immutable laws of social/political behavior, then the case study method gains greater scientific standing when compared to quantitative methods.
5/27/2007 10:16 AM
OK, you're right, it wasn't completely, 100% accurate statement.
which method(s) make(s) us as a discipline "better guessers"?
(1) Coming up with an articulable theory and clearly stating it for the record.
(2) Having an answer to the question "How would you know if your theory were wrong?"
(3) Comparing the predictions and anti-predictions of the theory to the world to see how well it holds up.
RE: 4:20 p.m.
I think that is fair statement.
coffee drinker said:
"Is the crux of your argument that when a hard scientist finds a correlation they are able to ascribe causation based on the fact that there are immutable laws of physics? And we in the social sciences have no immutable laws and therefore when we find correlation we are unable to ascribe causation? "
------------------------
Then ILP said:
"Those are key aspects of the argument being put forward."
-----------------------------
Alright lets keep going down the road here.
Once again, for the sake of argument, let me grant you the first part. That because of immutable physical laws when a hard scientist finds correlation that she can ascribe causation.
I think I would even agree with a weaker version of this. However, what part of the argument implies that in the absence of immutable laws we in the social sciences can not make causal claims? Is it your assertion that causal scientific findings can only truly be made when there are underlying immutable physical laws?
RE: 6:00 a.m.
What part of the argument implies that in the absence of immutable laws we in the social sciences cannot make causal claims? Is it your assertion that causal scientific findings can only truly be made when there are underlying immutable physical laws?
*****************************************************************
Allow me to answer your questions this way. The statistical models used by many in the social sciences were used to great effect in the physical sciences because of the immutable laws of physics. So what was found in the physical sciences is that statistical correlation can be a very, very strong indicator of how the immutable laws of physics work, and how matter and energy respond to these laws.
In the social sciences, without immutable laws, it is unclear what statistical correlation indicates. Because there is no claim* (or evidence) that statistical correlation in the social sciences is gauging the operation of immutable laws, it is just as likely that any such correlation involving social behavior is coincidence as it is indicative of causation.
From this follows the claim that scientific knowledge can only be attained through an understanding of immutable laws. Otherwise, we can document empirical phenomena, but cannot understand/"know" the causal mechanisms underlying empirical phenomena.
Now, do I subscribe to this view? It is definitely true that in the social sciences we can never attain the degree of certainty and understanding that is gained in the physical sciences. Nevertheless, I believe important social science knowledge is attainable, and we can gain insight into causal mechanisms. But we do have to be extremely conservative in our claims of causality. So, for instance, we have to be very careful in efforts to project our findings into the future. Moreover, claims of generalization are highly problematic. (Here I am referring to generalizations concerning all humans and their behavior.)
Perhaps most frustratingly, we have to accept that more than one legitimate interpretation of social phenomena can exist. Indeed, all we may be left with in the social sciences is "interpretation" and not "knowledge" in a scientific sense.
I would hold, however, that such interpretations constitute knowledge. Hence, I think the most appropriate tack for the social sciences to take is not to necessarily emulate the physical sciences (and their methods), but to stake out a different definition of science and knowledge in the study of social phenomena.
*Economists do claim to be working with the immutable laws of the market.
The statistical models used by many in the social sciences were used to great effect in the physical sciences because of the immutable laws of physics.
Yes, because physicists spend most of their time running OLS regressions, and astronomers are well-known for their reliance upon logit models.
Meanwhile, outside of the asylum, about the only field that makes really common use of regression-style statistics *AND* has a decent claim towards being physical sciences is biostatistics, but biostatistics (and its components like epidemiology) exists precisely because biologists can't write down simple mathematical models of their objects of study the way that physicists do.
RE: 7:47 a.m.
It is certainly true that in the physical sciences more sophisticated math is used than in most instances in the social sciences. But are scientists in the physical sciences familiar with (and use) linear regression? Absolutely! In any event, you are evading the key issues.
ILP is the most stubborn and disingenous person I have ever "met".
He should be a missionary in Africa.
Or a journalist at Fox News.
RE: 8:16 a.m.
Just because you are unaware of the tenets of science does not give you license to call anyone "stubborn" or "disingenous." If you have something substantive to say, then I encourage you to say it -- beyond that, please stay silent.
"you are unaware of the tenets of science".
Haha, yeah, right, ILP is truly well versed on "the tenets" of science.
RE: 8:16 a.m.
In truth, outside of your post I have not observed any posts that can be characterized as "stubborn" or "disingenous".
RE: 8:16 a.m. and 11:17 a.m.
If you find flaws in the ILP's reasoning, please point them out. I started as hostile to his/her view. I am presently on the verge of being officially convinced, however.
whatever happened to CC Banana? He/she was the only one keeping it real with this stuff.
re: 11:54.
I feel similarly to you. ILP has just about convinced me about methods in poli sci, and is also pretty close to convincing me that the earth is flat.
<< Home