Posts Tagged ‘Truth Versus Precision in Economics’

Truth Versus Precision in Economics by Thomas Mayer

September 12, 2009

TruthVersusPrecisionMy initial interest in Thomas Mayer’s Truth Versus Precision in Economics was spured when it was mentioned alongside McCloskey’s The Rhetoric of Economics in a footnote in a paper I read; the paper refered to it as a justification to accept unconventional p-values (probability of sampling error) in evaluating regression results.* Anyway, I picked it up at the library and was soon enthralled by Mayer’s sympathetic ideas.

The main claim in Truth Versus Precision is that economics is a victim of the principle of the strongest link, which leads to increased rigouization and decreased real-world relevance.

Mayer argues persuasively that economists has incentives to spend too much time on formalism, and that the formally explicit parts of arguments thus gets too much attention. Weaker parts of arguments are usually tended to by arm-waving. Strong, mathematically explicit arguments are subject to relatively much attention and are thus made stronger; weaker, implisit or verbal arguments receives less attention and remains weak. Further, the strength of a chain of arguments is often measured by the strength of the strongest argument, counter to the proverb that a chain is no stronger than its weakest link:

I call this procedure of focusing attention on the strongest part of an argument, and then attributing its strength to the entire argument, the ‘principle of the strongest link’ [p. 57**].

Mayer further suggests that economists preoccupation with formalism governs the prestige ranking of economics fields:

The prestige ranking of economics runs: first, formalist theory; second, empirical science theory; third, policy-advicing and data gathering, and fourth, history of economic thought and methodology [p. 46].

Mayer is a macroeconomist, and naturally parts of Truth Versus Precision discusses problems in macroeconomics. In particular, he argues that the foundations of new classical economics are questionable and concludes:

[N]ew classical theory is another example of the principle of the strongest link. Its advocates rightly take pride in the rigour of their deductive chains. But a rigorous deduction from a questionable premise, accompanied by no adequate tests of the conclusions, does not guarantee truth [p. 120].

Mayer also discusses the problems surrounding empirical testing in economics, for example that many focuses solely on Type I errors, that regressors with insignificant coefficients are excluded, problems with pre-testing of data, and confusions between statistical and substantive significance (see pp. 134 – 139). Finally, he discusses problems surrounding robustness tests (or rather, the lack thereof) (see pp. 142 – 147). He concludes the chapter on emprical testing accordingly:

[M]ost econometric testing is not rigorous. Combining such tests with formalized theoretical analysis or elaborate techniques is another instance of the principle of the strongest link. The car is sleek and elegant; too bad the wheels keep falling off [p. 149].

In the last chapter, Mayer discusses possible remedies. He calls for less abstraction and less formality; more replications and retests; proper use of statistical tests; care for data and awareness of anecdotal evidence; he wants journals to act as communication devices (not archives); critical evaluations of conflicting evidence; less focus on formal techniques in graduate training programmes; and more focus on writing skills.

All in all, I find many of Mayer’s arguments persuasive; they align with my feeling of unease when it comes to mathematical economics (note; I’m a mathematical economist myself). Some of Mayer’s critique also align with some of McCloskey’s critique. However, a professor at my school told me that Mayer was out-dated already in 1993 (the year of publication), and mentioned an article by Alexander Rosenberg from 1983 as evidence: Rosenberg discusses new classical economics. Notwithstanding, I still think there is something to Mayer’s critique, and as I said, it resonates with my own attitude towards economics. A more recent treatise discussing the very modern development of economics would be useful; have economics ridden itself of the principle of the strongest link? I need to find out.

* See p. 157; as far as I can see the only place in the text that actually argues for unconventional p-values, but not unconditionally.

** Page numbers refer to the paperback edition.

Related posts:

Science vs. Pseudo-Science

September 10, 2009

While discussing the selection bias in published economic research (the principle applies to most kinds of research, really, but some fields are more exposed than others) in his Truth versus Precision in Economics, Tom Mayer quotes the Richard Feynman, a Nobel Laureate in physics, on contrasting genuine science with pseudo-science:

But there is one feature … that is generally missing in … [pseudo-science.] It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty – a kind of leaning over backwards. For example, if you ar doing an experiment, you should report everything that you think might make it invalid – not only what you think is right about it; … [p. 141, italics in original].*

According to Mayer, a practice to only report the best of many results is reputed to be common among economists. I find this alarming. Of course, economists are aware of the possibility of a bias in reported results, and, as Mayer points out (p. 142), it leads to credible results receiving less attention than it deserves.

* See Mayer’s Truth versus Precision (p. 141) for the reference to Feynman.

A Vertical Division of Economics

August 26, 2009

Alright, this book (Truth Versus Precision in Economics, Tom Mayer) is so interesting that I’ve put Shakey aside for a while (in the middle of ‘Ohio’). I’m marking pages all the time.

In chapter 2, Two Types of Mainstream Economics, Mayer suggests to divide up economics vertically:

The fields that are often loosely referred to as ‘the sciences’ divide their subject matter not only horizontally by fields, e.g. biology and geology, but also vertically by the level of abstraction, e.g. physics and engineering, and physiology and medicine. In doing so they make room for different criteria at each level. An engineering paper need not be as rigorous as a physics paper. […] In the social sciences we divide fields horizontally, but not vertically, so that economics comprises mathematical formalism, empirical science work, and applications to specific practical problems. Hence we are tempted to apply inappropriate standards of rigour [p. 24, paperback edition].

On the following pages, he discusses the difference between formalistic and empirical economics, particularly in terms of rigour. He concludes chapter 2:

Although there is no sharp line of demarcation between hypotheses that are close to the observational level and those that are at a high level of abstraction, it is useful to classify the work of economic theorists into two categories, formalist theory and empirical science theory, because economists invoke widely different criteria in evaluating theories. Much misunderstanding is caused if one type of theory is judged by the criteria applicable to the other type [p. 36].

In the next chapter, he discusses whether there is too much of one type of economics and why that might be so:

We need both formalistic theory and empirical science theory. How much of each is more difficult to decide. While there are some reasons suggesting that formalist economics currently receives too little emphasis, there are stronger reasons to think the opposite is true. Professional self-interest, a belief that formalist economisits are much abler, and certain confusions all induce our profession to over-emphasize the importance of formalist theory relative to empirical science theory [p. 52].

Related post:

Economics: A Science or Not?

August 23, 2009

I came across this truly interesting book, Truth Versus Precision in Economics (1993) by Thomas Mayer. I’m still only on p. 16, but endnote 1 in chapter one provided fodder for my thinking about economics and science:

I see no purpose in discussing whether economics actually is a science. Philosophers have not succeeded in finding a criterion that distinguishes science from non-science […], and the question whether a field is an empirical science may even lack clear meansing […]. Fortunately, nothing hinges on whether one calls economics a science or not, and the question can be left to lexicographers. Knowing whether economics is a science would not allow us to decide whether  it should use the same methods as the natural sciences, since not all sciences necessarily us the same methods. What methods economics should use can be decided better by looking at specific methods and specific problems than by talking in general about ‘scientific method’. Similarly, knowing whether economics is a science would not allow us to say whether it provides answers that deserve a high degree of credence. The science of weather forecasting does not, while the non-science of history does [p. 8, paperback edition].

I agree that nothing really important hinges on whether economics belong to the (hard) sciences or not (what matters is that it is scientific). But, Mayer doesn’t seem to recognize that the English science has lost it’s propper meaning (here’s McCloskey’s explanation): Science means ‘systematic inquiry’ in any other language.

Related posts: