complexity

Loaded words and modeling

/ jdc / Leave a comment

Here is a nice review by Burton Malkiel of “Models Behaving Badly” by Emanuel Derman. The models of the title are from the world of finance: how are assets priced?

I am a layperson to the world of finance, so I find it difficult how to apportion “blame” for financial crises on faulty models or fraudulent inputs to them. Certainly history is littered with financial crises, so the influence of modern modeling alone cannot explain everything.

In any case, I just want to take this opportunity for a small lament that the beautiful act of modeling must be dragged through the mud by a financial crisis in this way. It would be fair to say that I am almost fanatical about the virtues of the concept and practice of modeling. I believe that modeling is inescapable. The world is complicated. Our senses deliver so much information, our mental apparatus must work so hard, that to process the world around us is to model. It is too much to ask that we understand everything; we have to understand a version of everything that is not so complex as the world.

This is also why economics works with models. We don’t have a scale replica of the world that we can play with to see how this affects that. We have to build a scale replica from scratch, using our best judgment to push insistently at the boundary between complexity (so that we can understand our model) and usefulness (so that we can make something from it).

In a way we are much luckier in economics than in finance. Progress in economic theory comes as our models are improved upon and refined, but we are more able to iterate forward because our models are not embedded in a Leviathan global finance industry that depends on their continued function. Creative destruction of old models is hard when the house comes down with them.

With all this in mind I want to highlight this passage from the review:

He sums up his key points about how to keep models from going bad by quoting excerpts from his “Financial Modeler’s Manifesto” (written with Paul Wilmott), a paper he published a couple of years ago. Among its admonitions: “I will always look over my shoulder and never forget that the model is not the world”; “I will not be overly impressed with mathematics”; “I will never sacrifice reality for elegance”; “I will not give the people who use my models false comfort about their accuracy”; “I understand that my work may have enormous effects on society and the economy, many beyond my apprehension.”

How many of these will I accept for economics? Certainly the first; the model is not reality. Certainly the second; math is helpful in model-building but is not the point of model-building. The fourth and fifth are hard to argue with.

The third I don’t like. Everything that we do must sacrifice reality. The test of a model is not its realism (a realistic model airplane would be no fun at all). All models are unrealistic because all models are wrong. Of course elegance is not the test of a model either, except that an elegant model is one that illuminates a relationship in a clear way by cutting to the heart of what matters.

Anyway, the point is that I think that “model” is not a dirty word. I feel possessive about “modeling” much the same way as I feel possessive about “rationality” – what they mean to me is important and wonderful and I hate to see them sullied by misrepresentation that stems from their overlap with the real-world ideas of modeling and rationality. I wish that all of the things like these could have their own words that are not borrowed from natural language.

Simplify, simplify

/ jdc / Leave a comment

I might just go ahead and quote myself:

“One of the principles of writing economic theory is to create a simplified abstraction of reality.”

This is from an article by Russell Jacoby in the Chronicle of Higher Education:

“The world is complicated, but how did “complication” turn from an undeniable reality to a desirable goal? Shouldn’t scholarship seek to clarify, illuminate, or — egad! — simplify, not complicate? How did the act of complicating become a virtue?”

This is quite clearly not an article about economics (phew). It goes to show how very, very different we’ve become from the other social sciences and arts. Yesterday I was talking about the development lab at MIT; would they say, “Ah, there’s a million and one things that affect the quality of education. I’m going for a drink.”? Of course not. Economics seeks to expose in the simplest possible terms the relationships around us. Indeed, the world is complicated; that’s why the MIT lab has to perform randomized trials to isolate the effects of programs. It’s why theorists create little models of the world.

Contrast this with this characterization from Jacoby:

“The refashioning of “complicate” derives from many sources…. [acaemics] will prize efforts not only to complicate but also to “problematize,” “contextualize,” “relativize,” “particularize,” and “complexify.””

In economics we want to know: what you’re saying, why you’re right, and what could make you wrong. That’s about it. One of the most valuable consequences of treating economics as a science is that we parachuted out of this borderline nonsense:

“They will denounce anything that appears “binary.” They will see “multiplicities” everywhere. They will add “s” to everything: trope, regime, truth. They will sprinkle their conversations with words like “pluralistic,” “heterogenous,” “elastic,” and “hybridities.” A call for “coherence” will arrest the discussion. Isn’t that “reductionist”?”

This explains a big part of the schism between positive economics and other social sciences; we are OK with leaving some things out if it helps. When it comes to the policy debate and the normative questions, we have to throw all the other stuff back in, but “it depends” is a conclusion acceptable in positive economic research only if you can tell me exactly how and why it depends.

Jacoby has the neat sign-off:

“The cult of complication has led — to alter a phrase of Hegel’s — to a fog in which all cows are gray.”

In economics, our judgment cows are gray, but our scientific cows are black and white.

Too complicated?

/ jdc / Leave a comment

One of the principles of writing economic theory is to create a simplified abstraction of reality. If the theory convincingly isolates an idea, it cannot be too simple; hopefully, the narrower the question, the simpler the theory can be written.

Economists therefore appeal to the “all else equal” assumption a lot. The oft-perceived superiority complex of economists is traceable to our willingness to use the “all else equal” clause to make our questions answerable, theoretically and empirically. If we want to write relevant economic models that investigate the link between A and B, we hold C equal; whether or not C would really be equal or relevant in reality, we can’t isolate the effect we’re interested in if we don’t figure out a way stop it from contaminating the abstraction.

It’s the same principle that underlies the ideal of “controlled experiments” in all science; empirically, if we want to figure out how A and B are related, I need to be careful to avoid finding an effect because a third factor C is involved. For example, there’s an important difference between “people who exercise more have a longer lifespan” and “people who exercise more also eat well, and people who eat well have a longer lifespan”. That’s well understood in statistics and empirics generally; there’s no reason why the same principle is not also needed when we use the theoretical standard of proof rather than the empirical standard of proof.

Why, then, is “economic theory” so amazingly bewildering? With very little exaggeration, we can claim that no great development in the science of economics has used very complicated techniques, even when math was involved, yet even to the technically competent a lot of economics research is very difficult to understand. Of course, if an economist could all find ground-breaking theory that can be represented in two lines, I’m sure she’d write it. Is the reason for the complexity an attempt to make average ideas look better?

Let’s be charitable and assume that’s not the case. I think that once we exclude the “obfuscation motive”, there are two possible reasons why economic theory is technically complex. One might be that the relationships being investigated are broader, that less is held equal, that we’re looking to more nuanced explanations. Another possible reason is, paradoxically, that theory gets more complex as the questions get narrower – the more we assume, the higher the complexity.

Why? Imagine I want to figure out the relationship between a person’s income and the number of hours that person does voluntary work. This is a question that asks about how people allocate a scarce resource, time. I might make an abstraction that says “if all people like both money and helping others, then people with higher incomes will spend more time helping others, while people with lower incomes will spend more time trying to earn extra money.” I might make an abstraction that says “people with more income work more so have less time to volunteer”. What assumptions lead to the first conclusion, and what to the second?

If I wanted to broaden my question, I might start including in my theory labor market conditions, the availability of volunteering opportunities, the peer pressure to volunteer, the social pressure to earn more money to buy a big car, and so on and so forth. That would certainly make my theory more complicated; whether or not it makes it a better theory than the one that kept all that stuff equal and abstracted from it is a matter of preference, but I’m sure it would be more difficult to understand.

The second way to make the theory more “complicated”, at least superficially, might be to keep all the same stuff equal, but to say “imagine the person cares this much about money and this much about volunteering; then someone with this income will volunteer this much”. The abstraction is getting more abstract; we are getting more and more specific about the conditions of our model, and we must use more specific techniques to, in particular, quantify the result.

What do we gain from this quantification, and what do we lose? Perhaps we can look at actual evidence on the link between income and volunteering, and compare it to the quantified prediction, but that only works if all else is equal in our evidence, too. A better justification is that we can get a theoretical idea of how big our effect is. However, as we get more specific we get more abstract; in this example, we’re getting more abstract about preferences, which are themselves unobservable. We’ve gone from “a person cares about money and volunteering” to attaching magnitudes to those cares.

The link between simplicity and usefulness is not just in the realism of the abstraction; it’s also in the procedure itself. Economic theory should be neither too broad or too narrow, but “just right”, whatever that means. Assume too little and we can’t figure out what’s really causing what; assume too much and you rest an entire argument on a special case. What’s the simplest model that explores the relationship I care about, and what’s the simplest model that shows what I want to show about that relationship?

Oh, and a practical suggestion: I’d love it if we all stopped writing ceteris paribus and used “all else equal”. What’s with the Latin?