Is This Good Science?

by johnmccreery

Taking off from what Drew writes in his last comment on “A Serious Question,” I offer the following quote from The SAGE Handbook of Case-Based Methods for our consideration.

From a trans-disciplinary perspective, what unites different kinds of cases, regardless of the discipline, is that all cases are complex and multi-dimensional objects of study. Furthermore, all cases are situated in time and space, as are the disciplines within which they might be situated. Arguably, therefore all cases, as objects of study, need to be described in an ever-increasing and changing variety of ways, and each of these ways may in fact be representing something ‘real’ about the object of study as well (2009: 141-142)

The proposition as I read it is that the traditional modernist gesture of scientific inquiry — abstracting variables from a complex situation for incorporation in a simplified but testable model — results in findings comparable to those of the blind men in the story in which they touch different parts of the elephant.  They thus wind up with totally different ideas about what an elephant is. The proposal offered here suggests that the blind men might talk with each other, bracketing claims that one or the other perspective is wrong, and consider the richer model produced when multiple views are superimposed. Personally, I find this approach attractive, especially for those of us in the social sciences and humanities, where our task is more often assembling fragments of information into compelling stories than testing well-defined hypotheses.

What do you think?


Advertisements

18 Comments to “Is This Good Science?”

  1. I think it reasonable to suppose that our perspectives on any given situation will be partial. Yet, if the two agree that they’re perspectives pertain to the same situation, and each is honest, then the task is to decide whether there are any real disagreements in their data. There should not be, except as artifacts of whatever inferential and descriptive processes are used to generate and describe that data. But there may be unresolved ambiguities. Imagine looking at a situation on a chess board, not with a bird’s eye view, but from the edge of the board. You will learn something about the situation on the board in the sense that you can eliminate possible configurations of pieces, but many possible configurations of chess pieces may not be ruled out given that information. To learn more, you will have to look at the situation from another perspective. This sort of view falls accords with ‘possible world’ notion of epistemic logic and information, weakly construed.

    This is complicated of course by the fact that we might not know what facts change from moment to moment.

    Completeness and Possibilities

    A logic function assigns an output from the set {0,1} to some set of input values also in {0,1}. For example, exclusive-or pairs inputs 00->0; 01->1; 10->1; 11->0. All together these make up a complete function table in the sense that the output for every possible combination of input values is defined.

    Suppose that I did not know that a given logic gate in some computer system was an exclusive-or gate, and I want to find out what logic function it instantiates by observing input output pairs. A two-input Boolean logic circuit might instantiate any of 16 different logic functions. If I observe that 00->0, then I can eliminate 8 possible functions (all those that 00->1). So far so good, but there is a basic problem. I can’t control the inputs to the function, so I don’t have any guarantee that my observational data set will be complete. My model may be a partial function. I would suggest that scientific experimentation may be seen as an attempt to ‘control the inputs’ to get as complete a picture as possible.

    One further point. If the objects of our descriptions and study exhibit regularities then it may very well be that none of our observational data will cover all logically possible state distributions. To return to chess: certainly it is possible to construct a configuration of chess pieces such that both the black and white kings are simultaneously in checkmate, but such a configuration of pieces is not possible in any properly played game of chess. Assuming that games of chess are our objects of study, it is necessarily the case that the empirical data we use to think about chess will never include such a situation. To think about these situations, we must draw on counterfactual reasoning.

    But what if the rules changed during our observational period, maybe several times? Our response I think must be a more fine-grained empiricism.

  2. Yes, we must look for disagreements. These may not, however, be grounds for choosing one perspective over another (the classic instance here is particle versus wave theories of light, with each accounting for data that the other does not). The right move may be a higher level theory that accounts for lower-level disagreements. I think here of Victor Turner’s noticing that what people do in rituals may contradict what they say about them. For Turner the contradictions in question were data for further analysis — pointers to fundamental conflicts in Ndembu social structure (matrilineal descent versus virilocal marriage in a society where ambitious men exploit both types of relationships to build their followings).

  3. When I introduce these issues to students I define a fact as something that looks the same or compatible from all perspectives. Is there actually such a thing? Isn’t the higher-level theory just another perspective?

  4. To put this another way: it generally seems to me that the humanities and even their most rigorous wing, the ‘social sciences’, produce what I think of as ‘optional’ readings – well-done, illuminating, productive perhaps but fundamentally dependent on our interest in attending to them. Claims to be something more are often made, of course, but tend not to stick outside the local knowledge systems that generate them. What would it take for an investigation in our inherently interpretive and situated fields to become not optional but essential?

  5. What do you mean by “essential,” white man? What, after all, is essential? Suppose we say necessary and sufficient. Necessary and sufficient for what? What is the purpose in relation to which necessary and sufficient are defined?

    If your purpose is to be absolutely sure and never have to change your mind (sounds a bit arrogant, that), that’s one thing.
    If your purpose is to satisfy the requirements for acceptance by a particular scientific or scholarly community (or a customer or your neighbors), that’s another thing.
    If your purpose is to write the code for the process that Jacob describes, you will look at data in one way.
    If your purpose is to examine an exorcism in detail (as I did in “Negotiating with Demons” (american ethnologist, 1995:1)) to explore the merits of looking at it in light of several current claims about ritual — (1) that magic is a performative act, (2) that magic is metaphor, and (3) that magic is use of a restricted code to assert unassailable authority — you look at data differently.

    The interesting thing to me about the quote from The SAGE Handbook of Case-Based Methods is the notion that looking at a case from any or all of these perspectives might reveal something real about it.

    But “real”? What’s that?

    Why not the essential, as described above, necessary and sufficient for the purpose at hand? The basic questions are always the same: What are we trying to do here? And how is what we are trying to do related to other people’s objectives?

    That works for me. Does it work for you?

  6. I wrote: “Assuming that games of chess are our objects of study, it is necessarily the case that the empirical data we use to think about chess will never include such a situation. To think about these situations, we must draw on counterfactual reasoning.”

    I should have wrote “…we must either draw on counterfactual reasoning or upon experimentation.” It seems to me that experimental controls may be a way of circumventing this partiality of naturalist perspective.

  7. John, I agree with all of this and have said so many times. But what this means is that what counts as ‘good science’ is only going to be minimally amenable to specification a priori, and therefore it’s only going to be briefly and minimally useful to ask that question absent cases. As you say the basic questions are always the same, and the rest emerge out of the specific dialectic of specific theory and practice. And since you know this, and I know you know this, and you know I know this, yet this seems to be the conversation you’re proposing, I’m trying to play along, which is why I threw out the question about essential knowledge. Which I’m awfully glad you threw back at me. ;-P

    As for what’s real, perhaps the Greenland icepack is attempting to teach us something about that by melting, raising sea levels, pushing the Gulf Stream away from the British Islands, changing rainfall patterns across Europe and hurricane-generating convection off the coast of Africa. Our responses to this kind of reality, at Copenhagen for example, are also real but maybe in an interestingly distinct kind of way. And the monster in my closet is just as real as it needs to be to frighten the crap out of me right up until I turn on the light, and by doing so melt not just the monster but a fraction more ice.

  8. Sorry, if I have been belaboring the obvious. These issues are much on my mind because I am now wrestling with how to combine three different perspectives on my current project, exploring the social networks of top Tokyo ad creators. I have

    1. Social network analysis of winning ad credits. The results are what the mathematicians predict for networks on this scale: giant components, giant bi-components, power law distributions of key centrality measures. Here I am talking about an uncanny degree of precision. The social physics works.
    2. Predictable network properties do not, however, explain everything. There are many features of the networks that seem best explained by economic and institutional factors: a highly oligopolistic Japanese ad industry, radically changing shares of total ad spend with the rise of TV and decline of print media during the ’80s, ’90s and ’00s, followed by a major shift to spending on collateral, e.g., direct mail, and now the Internet.
    3. What the central figures in the networks I analyze have to say about such topics as creativity, leadership, teams, teamwork, the shifting focus in advertising from copy to visuals, etc.

    Consider, for example, the case of Sasaki HIroshi, who has won more Japanese advertising awards than anyone else in history. Is his success due to his character, talent or published ideas about creativity, etc.; his working for Dentsu, the biggest agency in the business; his starting his career at a time of spectacular industry growth; a series of spectacularly talented subordinates (about whom all of these questions could be asked); or network dynamics generating a power curve in which, purely by accident, he happens to be in the right place?

    Any and all suggestions about how to theorize this mess are welcome.

  9. John, I feel like the rubber is now on the road. This is really, really interesting.

    It looks from your sketch like the ‘agency’ in the situation is pretty much subordinate to the big deterministic network numbers. Maybe Sasaki was just in a fortuitous nexus, as you say. Are you thinking of 3. then as a sort of false consciousness analysis; or does it show how the big numbers happen grain by grain; or is there stuff the social physics doesn’t quite capture…?

  10. No. This isn’t a false consciousness analysis. I haven’t met Sasaki (“Hiroshi” is his first name); but from what he has written and said in interviews, I’d judge that he is highly conscious of what he does and does very well, indeed. What pops into my mind is a teaching story I made up after reading Tom Peters’ Pursuit of Excellence, in which Peters argues that successful companies are both constantly gathering information and also willing to take risks.

    I ask students to imagine a beach. At one end there is someone building the most amazing sand castle you have ever seen: a thing of beauty until the tide comes in. At the other is the fellow who drives up, tears off his clothes, and rushes into the water, oblivious to storms, riptides, sharks, jellyfish or other dangers. He is likely to drown. In the middle is Surfer Dude. You watch him as he walks across the beach carrying his board. You note that he is in great shape and appears to know what he is doing. He checks the weather. He checks the water. When he spots a good wave he paddles out to meet it. You can tell that he’s had a lot of practice doing this sort of thing.

    Not even Surfer Dude is always successful. He can miss a wave. He can crash out. But when he does catch a big wave just right, he can have an incredible ride. The network physics and the economic and institutional stuff? That creates the waves. Guys like Sasaki? They’re Surfer Dudes.

    Still stuck in metaphor; but that’s where my thinking is now.

  11. John:

    You may be interested in some of the work by Camille Roth on the co-evolution of epistemic communities. I’ve been meaning to bring up his work to you, though I haven’t had time to digest it myself. Too much on my plate.

    An Extended Abstract: http://camille.roth.free.fr/travaux/roth-uksocnetworkshop.pdf
    Roth’s PhD thesis: http://camille.roth.free.fr/roth-thesis.php

  12. Thanks, Jacob. Roth looks really interesting. Now, if can get through this pays-the-bills work I have to do today….

  13. One of my favorite students a few years ago was a Sasaki – Ryuta. Smart, interesting guy. I was more heavily into my self/identity project at the time and he filled me in on all the different relational ways to say ‘you’ in Japanese.

    I love the Surfer Dude metaphor. It looks to me like what Weber called elective affinity and Bourdieu called habitus or a feel for the game. There’s so much situated knowledge packed into that feel. What sorts of things does he know? How does that direct his attention?

    Jacob, that Roth thing looks great; I’ll probably get to it right about when you do….

  14. Ha Ha. I suppose I deserved that.

    Actually it does look great, and I hoped relevant to John’s interests too. That is why I recommended it. I have read some of Roth’s work, but his work is not particularly relevant to anything I have to do, and so tackling heavy-duty pieces of literature like his dissertation is really out of the question until I can put away this little comp. sci. thesis thingy I’ve got to do.

  15. Taking a break, I pause for a look at Frames/ing and find Kvond saying,

    We do not look at ourselves in a mirror (of consciousness or any other), and then make adjustments in idea. Rather, our concrete “position” is itself a positional change. This goes down into a radical sense of what (self) affirmation is, a non-reflective (relatively) autonomous embrace which includes that which cross-currents our own being, propelling us out to mutualities.

    I think of this in relation to Surfer Dude and G. H. Mead’s notion that we learn to see ourselves, the “me,” the object of reflection, as others see us. Kvond pushes me toward realization that none of us ever see ourselves quite as others see us. We we look in mirrors, the image is reversed. Photographs capture only instants. We respond to how others act toward us — but see ourselves through their eyes? No.

    Now back to work.

  16. I know I’m coming in late on this, but I thought I should comment, given that my comment ostensibly kicked this off…

    I must confess to being a bit confused about the thread. Perhaps because this is because I’m not a social scientist. But I guess my response to the post would be to inquire into the sense of good in ‘is this good science’, and also to ask if science is concerned with good science, or just getting results (and delineating the difference between these two)?

  17. Good question, Drew. How would you distinguish between them?

  18. With a tip of the hat to Neuroanthropology, which provided the link to this discussion of the endless crisis of anthropology.

Leave a Reply!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: