Critical thinking

Julian Sanchez has been thinking about why discussions of climate change rely so heavily on “countless experts agree” type arguments, and makes a good point about evaluating experts' arguments.

Come to think of it, there’s a certain class of rhetoric I’m going to call the “one way hash” argument. Most modern cryptographic systems in wide use are based on a certain mathematical asymmetry: You can multiply a couple of large prime numbers much (much, much, much, much) more quickly than you can factor the product back into primes. Certain bad arguments work the same way—skim online debates between biologists and earnest ID afficionados armed with talking points if you want a few examples: The talking point on one side is just complex enough that it’s both intelligible—even somewhat intuitive—to the layman and sounds as though it might qualify as some kind of insight. (If it seems too obvious, perhaps paradoxically, we’ll tend to assume everyone on the other side thought of it themselves and had some good reason to reject it.) The rebuttal, by contrast, may require explaining a whole series of preliminary concepts before it’s really possible to explain why the talking point is wrong. So the setup is “snappy, intuitively appealing argument without obvious problems” vs. “rebuttal I probably don’t have time to read, let alone analyze closely.”

If we don’t sometimes defer to the expert consensus, we’ll systematically tend to go wrong in the face of one-way-hash arguments, at least our own necessarily limited domains of knowledge. Indeed, in such cases, trying to evaluate the arguments on their merits will tend to lead to an erroneous conclusion more often than simply trying to gauge the credibility of the various disputants. The problem, of course, is gauging your own competence level well enough to know when to assess arguments and when to assess arguers.

To that last point, knowing that in consequence of being a pretty smart cookie I tend to overestimate my competence for this sort of thing pretty frequently ... and noticing that folks significantly less competent than myself commonly have a tendency to overestimate the value of their judgment much more than I do ... this is a disconcerting observation.