Methods of Discovery

by johnmccreery

Having ignited an interesting conversation with a quote from Seth Godin, I now offer one from another of my favorite authors Andrew Abbott, stimulated by the observation that, like Godin, Abbott is all about empowering people to move beyond constraints internalized in school. The following is from “To the Reader,” a preface to Methods of Discovery: Heuristics for the Social Sciences, 2004, New York: W.W. Norton, xi-xii,

What does it take to have something to say? It takes two things. The first is a puzzle, something about the social world that is odd, unusual, unexpected, or novel. The second is a clever idea that responds to or interprets or solves that puzzle. Everything else—the methods, the literature, the description of data—is really just window dressing. The heart of good work is a puzzle and an idea.

Good work, says Abbott, requires creativity. That said, however,

Creativity cannot be taught. As John Dewey put it, “[N]o thought, no idea, can possibly be conveyed as an idea from one person to another” (1966:159). We can teach others the backgrounds, the conditions, and the origins of an idea. But if we tell the the idea itself, they don’t really have it. To really have it, they must make it inside themselves. Jane Austin put it a little more bluntly in Pride and Prejudice: “We all love to instruct, though we can teach only what is not worth knowing.”

Now two words in the title of the book snap into focus: “Discovery” and “Heuristics.” The goal of good work is discovery, but there is no recipe to be followed. There are, instead, heuristics, ways of approaching problems that have proven productive before. They might be worth a try.

*Dewey, J. [1916] 1966. Democracy and Education. New York: Free Press.

Advertisements

19 Comments to “Methods of Discovery”

  1. What does it take to have something to say? It takes two things. The first is a puzzle, something about the social world that is odd, unusual, unexpected, or novel. The second is a clever idea that responds to or interprets or solves that puzzle. Everything else—the methods, the literature, the description of data—is really just window dressing. The heart of good work is a puzzle and an idea.

    I’m all for heuristics and creativity. But I’m a little troubled by the wiftiness of this particular formulation. The blogosphere is an object lesson in the very uneven quality of things people have to say when all they’ve got is a puzzle and a clever idea. When I was 16 I and my friends were full of puzzles and clever ideas. That’s a start, but now I’ve got methods and some density of both experiential and literate knowledge that help me sort real puzzles from dead ends and orient me toward quality investigation likely to yield robust data. Window dressing? That stuff is what separates responsible practitioners from pub gurus.

  2. (I realize that my concern may well be addressed further in as Abbott cashes out what he means by heuristic.)

  3. Carl, I don’t think Andrew Abbott would disagree. In fact, I am having a bit of a giggle at thinking of a genuine intellectual heavyweight and author of books that are very densely argued, indeed, as “wifty.”

    Abbott is one of the few authors I know who is perfectly comfortable across the whole spectrum of social science methods, which, in Methods of Discovery he divides into five broad categories: ethnography, history, standard causal analysis (linear regression based on large-sample statistics), small-n comparisons, and formal simulations (mathematical and computer modeling). He uses the term “heuristic” quite precisely, to designate a method or collection of methods that, while imperfect, advance knowledge in the direction of better approximation to what is going on.

    What he does very well, indeed, in my view is avoid the sort of fetishism that asserts that either qualitative or quantitative methods is THE ANSWER. He is also shrewd in appraising what counts as good work in the categories he discusses.

    I was particularly struck by his remark that what we look for in an ethnographer is evidence that he or she was, in fact, an unbiased and accurate observer of what the ethnography reports. This is often a matter of persuasive detail. He cites, for example, Evans-Pritchard’s noting in Witchcraft, Oracle and Magic Among the Azande that, while he was living among the Azande, he, too, used the poison oracle to organize his activities and found it as good as any other system he had ever used.

  4. Sometimes I wish I could get past the visceral dislike I have for Abbott.

    Another thing, though: while it becomes a kind of infinite mirrored hall, one must also ask what makes something seem to be a “puzzle.” We take the rules of a given language game for granted, we train our children in those rules–but someone raised in a different language game will see things differently. Nothing about the social world is inherently odd, unusual, unexpected, etc.; one perceives it that way because one’s own usual social world, habitus, language game, wtf-ever, leads to a particular set of expectations.

    Without having read the rest of what he’s doing, and possibly misinterpreting, he seems to be privileging the Scientist viewpoint.

  5. privileging the Scientist point of view

    Given that the man is writing a textbook about how to do scientific research, what else should he be doing?

    We take the rules of a given language game for granted,

    Not if you’ve learned a foreign language, traveled a bit in either physical or social space, or grown up in a time when the rules are changing and people are always fighting about which ones do and don’t apply.

    What planet do you live on?

  6. It intrigues me that comments so far have focused on the proposition that good work is finding an interesting puzzle and a clever idea. I was expecting a bit more blowback from

    As John Dewey put it, “[N]o thought, no idea, can possibly be conveyed as an idea from one person to another” (1966:159). We can teach others the backgrounds, the conditions, and the origins of an idea. But if we tell the the idea itself, they don’t really have it. To really have it, they must make it inside themselves. Jane Austin put it a little more bluntly in Pride and Prejudice: “We all love to instruct, though we can teach only what is not worth knowing.

  7. Dewey was just right about that. No blowback from me.

    Re: puzzles and language games, I’m going to channel here what NP of Rough Theory has been saying for the last little while about Marx’s critical method, which is that it’s all about shifting perspective to move around and through things. One perspective will never do it, although mostly we live as though it will.

  8. It’s all about shifting perspective to move around and through things. One perspective will never do it, although mostly we live as though it will.

    Bingo! One of the things I like about Methods of Discovery is the way in which the attitude it promotes encourages thinking about all the possible links and loops connecting the five broad categories. When I was a graduate student the only plausible path, it seemed to me, was to start with ethnography to explore an unknown space then develop hypotheses for statistical testing, and even that meant straddling the camps of the Quals and the Quants, who tended to despise each other. Something like my current project, which involves data mining and social network analysis pointing to structures in need of closer ethnographic and historical study, was, if not unthinkable, utterly impractical.

  9. Idly I wonder what kind of overlap there may be between Qual/Quant and Fox/Hedgehog. Are Quals Foxes and Quants Hedgehogs? Or do Qual Hedgehogs fight Quant Hedgehogs, while Foxes use whichever tools work to get at the things they want to know?

  10. If we accept Abbott’s analysis in Chaos of Disciplines, Qual/Quant is one of the perennial divisions that reoccur recursively in the fractal process by which disciplines emerge. Suppose, for example, that we have two camps, the Quals and the Quants. If both grow sufficiently large, the Qual/Quant split will reappear inside them both, with pure Quals in opposition to Quals who embrace some Quant methods and pure Quants opposed to Quants who embrace some Qual methods. The “if sufficiently large” is important. There are always groups forming that try to distinguish themselves by some unique combination of problems and methods, but most will rapidly disappear. Only those in which the founders manage to recruit and train successors will carry on, and only those that attract large numbers are likely to become established disciplines.

    That said, it seems to me that Qual/Quant and Fox/Hedgehog are independent dimensions that lie orthogonal to each other. My old college roommate John Baldwin, who is now an Emeritus Professor of logic and attends meetings where less than a dozen people have the slightest idea what is being talked about is, professionally speaking, a hedgehog. But so is Joan Piggot, another friend, who is one of the world’s leading specialists on Japan’s ancient history (roughly the period between the first Chinese records around 300 a.d. and the founding of the current Japanese empire roughly a thousand years later). I count myself a fox, since I do research and publish on both Chinese religion and Japanese advertising, have spent a lot of time on the languages necessary for both, and am now up to my ears in social network analysis, working on a project that is both Qual and Quant.

  11. Given that the man is writing a textbook about how to do scientific research, what else should he be doing?

    Well, same comment as Carl’s above, more or less: He is apparently writing a textbook about how to do scientific research, and, simultaneously, is saying that the methods of same are “window dressing,” that what is important is a puzzle and an idea. Further, according to the quote you chose, he also says that creativity is important, and that it can’t be taught. In other words, all that schooling pales in worth, when stacked next to Ideas, and Puzzles, and Creativity.

    What planet do you live on?
    Well of course rules are being contested–but not ALL rules, and not ALL the time; one cannot use Humpty Dumpty’s version of the meaning of words (they mean whatever he says they mean), else even a simple blog comment is impossible. But observing or participating in a fight about something, or absorbing those shifts in perspective that come from leaving the metaphorical nest, and having the foggiest clue what to do about the conundra those experiences bring up, isn’t just about puzzles and ideas. And even then, there are still a buttload of things that one will take for granted, at least some of the time, in at least some contexts (and possibly many).

  12. there are still a buttload of things that one will take for granted, at least some of the time, in at least some contexts (and possibly many).

    That said, it makes little sense to leap to conclusions about a 282-page book on the basis of two paragraphs from a preface. Especially, in my view, if one starts with a personal animus toward the author.

    If anyone would like to learn more about the content of the book, I will do my best to oblige, while urging you to read it for yourself. If not, I’ve got plenty of other things on my plate just now.

  13. John, I apologize; personal animus notwithstanding, I was, in fact, trying to engage with what you quoted, and I agree that the questions raised are worth raising. Really, my non-personal comment was much like what I take Carl’s to have meant: I think it’s a valid question to ask whether a puzzle and an idea and creativity–the latter of which cannot be taught–ARE what it takes, that the methods, description, etc., are, in Abbott’s own words, “window dressing.” Those first two paragraphs, from the preface, can be taken as a “manifesto,” insofar as any preface can or should be taken that way (which may be only in a trivial way)–and Carl’s second comment holds as well; Abbot may well cash that out in useful and interesting ways.

    Let me explore the creativity notion this way: I have learned several arts/crafts over the years, and am in the middle of learning another one now. Am I a “naturally” creative person? Well, maybe, a little–but it’s also the case that the process of learning, and using, the methods of a given art/craft enable one to consider doing things–to be creative–in ways that are not possible to even consider without that knowledge. Doing interesting things with laminated dough is only possible once one knows how to make laminated dough, or, to use the current example, creating ceramic tiles is only possible when one knows the methods to be used. Knowledge of method can lead to experimentation of particular sorts and within particular bounds, and possibly to results that, ultimately, are “creative” in some relevant sense of that. Thus, in some fields, at least, the methods and literature are not, in fact, window dressing, but, instead, are the tools one uses. Is scholarship analogous, or not? Do the tools that we learn in school enable us to ask questions, to identify puzzles and have ideas? And the title of the book, as you note, suggests that, despite the preface, yes, tools (broadly construed) DO matter. I’ll be interested to know whether or how you/Abbott think that plays out.

  14. Thanks Narya, that’s well said and just what I was wondering. I see the danger that methods become procrustean beds, and hope that Abbott is suggesting an informed but more flexible approach. But the ‘manifesto’, as you say, looks like it may be a rhetorical overstatement.

    Just to add an example, I built mobiles for a long time, working up from a primitive point-balance-and-hang approach, before I gradually got to where I could attune to what Calder was up to. There’s a formula, but it’s open to loads of theme and variation. It’s what I might call a domain of creativity, and entering it was impossible without a certain level of methodological accomplishment. I’m now limited by the resources of my shop and lack of skill with metals (cutting, shaping, welding). Such factors of material and artisanal culture are part of the story of creativity, in this and many other fields.

  15. What we just said seems consistent with this from Abbott: “We can teach others the backgrounds, the conditions, and the origins of an idea. But if we tell the the idea itself, they don’t really have it. To really have it, they must make it inside themselves.”

  16. Re: Calder, you can see him working his way toward the mobile form we’re familiar with over a period of years of experimenting with balance, materials and so on. He pulled in stuff from a variety of existing domains, but it’s still fair to say that in elaborating and configuring his sources he invented a new domain of creativity. I’m inclined to say that this is a different level of creativity that mine in tromping around in the field he staked out.

  17. What I like best about Abbott is the reflexive and descriptive stance he adopts toward methodology. Chapter 1 is titled “Explanation.” But instead of the usual move, assuming a definition and spelling out its consequences, which are then taken as given throughout the rest of the book, he asks his reader to consider the practices that social scientists refer to when they say “explanation.” There are, he observes–and this, I note, is an observation by someone who has closely studied the history of sociology and written several heavy books about it–three types of explanations, which he labels pragmatic, semantic and syntactic respectively.

    The pragmatic explanation identifies a key variable, a lever that can be used to produce some desired effect on the world. Formally speaking, the explanation is a function y=f(x), where modifications in x produce corresponding changes in y.

    The semantic explanation is, in effect, a translation. Something that may, at first, appear strange is mapped onto a semantic space whose terms the explainer takes for granted. Formally speaking, the explanation is the proposition that x=y, where x is initially unknown but the explainer assumes a given meaning for y.

    A syntactic explanation describes a process unfolding over time. A familiar example is a linear narrative in which x is described as the outcome of a series of steps y[1], y[2]…y[i], where y[i] denotes an event occurring at time i.

    Having considered these three possible types of explanations, Abbott turns to methods and notes that they can be characterized in three different ways, in terms of data gathering, in terms of analytic technique, and in terms of the number of cases considered.

    There are four basic approaches to data gathering.

    1. Ethnography– gathering data by personal interaction
    2. Surveys–gathering data by submitting questionnaires or conducting formal interviews
    3. Record-based analysis–gathering data from organizational records
    4. History–using old records, surveys, even ethnographies

    There are three basic approaches to analysis

    1. Direct interpretation–analysis by individual reflection and synthesis
    2. Quantitative analysis–using one or more standard statistical methods
    3. Formal modeling–creating a formal system mimicking the world and using it to simulate reality

    There are three basic approaches to choosing the number of cases=n.

    1. The case study, n=1
    2. Small n
    3. Large n

    Multiplying 4x3x3=36 possible combinations, of which the explanatory programs (here program means a particular combination of choices from the three lists) associated with, for example, anthropology (ethnography), history (history), quantitative sociology (standard causal), historical sociology/comparative politics (small n comparison), and economic theory (formal modeling) account for only five. Why only these five have become the major alternatives for social science research is, of course, an interesting question for further study.

    At this point, I have, however, only briefly outlined Chapter 1 “Explanation”

    Chapter 2 “Basic Debates and Methodological Practices”
    Chapter 3 “Introduction to Heuristics”
    Chapter 4 “General Heuristics: Search and Argument”
    Chapter 5 “General Heuristics: Description and Narration”
    Chapter 6 “Fractal Heuristics,” and
    Chapter 7 “Ideas and Puzzles”

    All of these chapters build on the basic framework articulated in Chapter 1.

  18. I see. Yes, I like how he’s got the schema set up so it can accommodate a variety of priorities and permutations. I assume the following chapters use the rubric of heuristics to discuss the conditions under which particular constellations of approach might be chosen and the consequences of those choices?

  19. I see. Yes, I like how he’s got the schema set up so it can accommodate a variety of priorities and permutations. I assume the following chapters use the rubric of heuristics to discuss the conditions under which particular constellations of approach might be chosen and the consequences of those choices?

    That’s a very nice summary….but there’s more.

    in Chapter 2 “Basic Debates and Methodological Practices,” he introduces two other important ideas.

    A. The basic explanatory programs don’t just differ in method. Ontologically speaking, they see the world in radically different ways.
    B. As a result of A, they are not related as points along a gradient, e.g., the one I was taught, with exploratory research at one end and hypothesis testing at the other. Their relationship to each other is circular and resembles that of rock, scissors, paper in the familiar children’s game, where rock breaks scissors, scissors cut paper, and paper covers rock.

    Since we’re dealing with five programs, and I am an East Asianist, I think an even better metaphor would be the five elements/agents in traditional Chinese cosmology: Earth, Wood, Fire, Metal, and Water, which both generate and suppress each other in different combinations.

Leave a Reply!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: