Models, Phenomenology, and All That

There is a profound misunderstanding by some people of the fundamental nature of models, and more generally of quantitative finance and risk management, from accountants to professors to players in the world of finance to the general public. A better analysis and attribution of some causes of the current financial crisis – and perhaps helping to prevent future such crises - would be obtained by a better understanding of what financial models really are. Emanuel Derman and Paul Wilmott have a good Manifesto, to which I subscribe and effective already pre-signed in my book. Below are some quotes from the book (ref) that indicate my Philosophy of Models, the basic lines of which I have preached ever since coming to the Street in the late 80’s, and written long before the current crisis. The bottom line is that, in spite of the fact that Models, Quantitative Finance, and Risk Management may resemble “hard science”, and while we should and do try to be as scientific as possible, the real deal is that what we have always had is what physicists call “phenomenology”. This does not mean that the models are useless. We always need qualitative judgment, but we also need models. What does all this mean? What are some implications? Read on for some thoughts.

Ref: Excerpts from Quantitative Finance and Risk Management, A Physicist’s Approach: Pp. 7-8 (Ch.2); Pp. 415-420 (Ch. 32); Pp. 421-424 (Ch. 33); footnote 9, page 508 (Ch. 42).

“

Why is Quantitative Finance not a Science?

In science there is real theory in the sense of Newton's laws (F = ma) backed by a large collection of experiments with high practical predictive power and known domains of applicability (for Newton’s laws, this means objects not too small and not moving too fast).

In contrast, financial theoretical "postulates", when examined closely, turn out to involve assumptions, which are at best only partially justifiable in the real world. The financial analogs to scientific "experiments" obtained by looking at the market are of limited value. Market information may be quite good, in which case not much theory is needed. If the market information is not very good, the finance theory is relatively unconstrained. Finance computer systems are always incomplete and behind schedule (this is a theorem).

.

Quantitative Finance is Not Science but Phenomenology

The situation characterizing quantitative finance is really what physicists call "phenomenology". Even if we could know the "Newton laws of finance", the real world of finance is so complex that the consequences of these laws could not be evaluated with any precision. Instead, there are financial models and statistical arguments that are only partially constrained by the real world, and with unknown domains of applicability, except that they often break when the market conditions change in an extreme fashion. The main reason for this fragility is that human psychology and macroeconomics are fundamentally involved. The worst cases for risk management, such as the onset of collective panic or the potential consequences of a deep recession, are impossible to quantify uniquely—extra assumptions tempered by judgment must be made.

.

What About Uncertainties in the Risk Itself?

A characteristic showing why risk management is not science deals with the lack of quantification of the uncertainties in risk calculations and estimates. Uncertainty or error analysis is always done in scientific experiments. It is preferable to call this activity "uncertainty" analysis because "error" tends to conjure up human error. While human error should not be underestimated, the main problem in finance often lies with uncertainties and incompleteness in the models and/or the data. Risk measurement is standard, but the uncertainty in the risk itself is usually ignored.

In finance, there is too often an unscientific accounting-type mentality. Some people do not understand why uncertainties should exist at all, tend to become ill tempered when confronted with them, and only reluctantly accept their existence. The situation is made worse by the meaningless precision often used by risk managers and quants to quote risk results. Quantities that may have uncertainties of a factor of two are quoted to many decimal places. False confidence, misuse and misunderstanding can and does occur. A fruitless activity is attempting to explain why one result is different from another result under somewhat different circumstances, when the numerical uncertainties in all these results are unknown and potentially greater than the differences being examined.

.

Summary of Model Risk

We start with the obvious comment that models are now an indispensable part of modern finance. Securities and derivatives require model pricing. Therefore, models are indeed indispensable. Finance could not live without them.

Nonetheless, in spite of the best efforts of many talented and smart people, model risk is constantly present at some level and is due to many causes. One risk is the variability in model assumptions, none of which can be proved in any rigorous way . Some important effects on prices can be modeled only imperfectly, if at all. No financial model has the status of a “law of physics”, even if physics-based diffusion models and other concepts are used. Further, there is no “best” model, regardless of whose ego is involved. Different firms can and do have different models for the same instrument.

Model risk is hidden unless model-to-model or model-to-market comparisons are made. The risk is highest for illiquid, long-dated options. For highly liquid instruments, models are standardized with slight variations. Substantial losses due to model risk have occurred even for plain-vanilla products, however .

Model risk includes the risk of using approximate or inappropriate parameter types, or using the model in inappropriate parameter regimes. Models are used in practice to parameterize securities in some approximate way, and are usually only to be trusted for some short extrapolation from the region where market data are available. These parameters include maturity length, strike values for options, etc.

The intentional use of inaccurate parameter values is a separate problem .

Numerical approximations are unavoidable and are a function of available time and resources, but they can lead to difficulties.

Part of model risk lies in the pitfalls of software development. See Ch. 34. A host of mundane but important issues exists: coding errors, computer malfunctions, misinterpretations, communication snafus, inconsistencies, etc. Anyone who wants to get an idea of the difficulty is invited to sit down at the computer and give it a try .

Model-generated hedging predictions are more problematic than pricing, since hedging involves taking differences of prices under changing market conditions. Models differ more in the hedges than in the prices.

Model risk issues have arisen in a number of contexts. The interested reader is invited to consult the literature and the references .

.

Model Risk and Risk Management

Models stand at the cornerstone of risk management. Models characterize the behavior of financial instruments under different possible environments. This information is used to determine the risk of these instruments, and thus of departments, and ultimately of the corporation with respect to the markets. We have exhibited a variety of model calculations. We need to understand the model limitations. These limitations translate into a risk associated with the very models used for assessing risk. Model risk results from model limitations.

.

Liquidity Model Limitations

There is a variety of other problems not included in models. Effects related to supply and demand, the trading volume, and the time needed to sell a security are lumped together into "liquidity", and there are no good way to model these effects. Often they are just left out of the model price.

Bid-offer spreads are a related issue. If there is only a one-sided market so that (for example) the selling price is not known, then these spreads must be estimated independent of the model algorithm.

.

Which Model Should We Use?

Because there is no real theory of finance in the sense of physics, financial models are not unique. Different institutions, especially for illiquid financial products, often use different models. If one model is used in place of another, the differences in the values of the securities and the differences in the sensitivities with respect to movements in the underlying variables become an issue in risk management. For example, if one model reports the interest-rate dependence of a partially hedged position is near zero while another model just as sophisticated and defended with at least as much exuberance reports that the interest-rate dependence of the same position is large, which statement do you believe?

Sometimes a proprietary desk model is used for trading, and another model with simpler assumptions but widely used on the Street is used for corporate risk management reporting. Which model should be used to measure risk? The real answer is that there is model risk. Different models give different results. Therefore, there is an uncertainty in risk reporting due to the very existence of different models. A corporate goal should be the quantification of this model risk, creating an uncertainty of risk management itself.

.

Psychological Attitudes towards Models

The psychological attitudes toward models are not to be ignored. People who do not understand the limitations of models ask for the "best" model, and some people who should know better may believe they have the "best” model. Some people trust the models to such an extent that if their model disagrees with the market they assume the market is "wrong" and will eventually agree with the model. Sometimes this attitude pays off and sometimes it results in disaster. Sophisticated players understand and even fear the limitations of the models, sometimes using them only as a guide in difficult markets for illiquid products.

.

Model Risk, Model Reserves, and Bid-Offer Spreads

Models used for risk management are themselves risky to some extent. A corporate reserve could be taken to account for this model risk. This is difficult to convey to accountants, who want to know exactly how much the model risk is and exactly when or under which conditions they should apply the reserve. Because model risk will really show up when relatively illiquid positions are sold in difficult market conditions under pressure at someone else's model price, the risk is hard to quantify. Still, model risk is not zero and may be very large.

Alternatively, if known, model risk can be used to estimate part of the bid-offer spread for illiquid products.

.

Model Quality Assurance

The best way to quantify model risk is through a model "quality assurance" (QA) program (c.f. Ch. 33). Model QA now exists in most large financial institutions, although model risk is generally only determined in an incomplete fashion. Model assumptions and procedures are documented. Sensitivities to different parametric assumptions can be examined. Models can be assessed and compared.

.

Models and Parameters

Because there is no financial model that proceeds from first principles that are unambiguously correct, models are largely driven by parameters. These parameters are chosen through a combination of somewhat conflicting goals. The parameters are chosen such that the model into which the parameters are placed produces prices that, at least approximately, fit selected market data. The difficult problem is for cases where there is little or no market information. Models differ partially because the market constraints placed on the models can be chosen in different ways. The number of parameters is a compromise between fitting the known market prices and unwieldy complexity.

Having chosen the parameters to fit the market approximately, the models can be viewed essentially as providing an extrapolation or interpolation methodology. Thus, if a deal comes up that has parameters not currently quoted in the market, which is often the case for over-the-counter deals, the model is used to derive a price for that deal. The models, through extrapolation or interpolation, price illiquid instruments in a portfolio.

It should be emphasized the types and numbers of parameters in reality form an integral part of a model. In a profound sense, the parameters cannot be separated or isolated from the assumption of the underlying dynamics and the implementation of the mathematics through some computer algorithm. For example, if one volatility is used to describe a diffusion process rather than several volatilities, this is a model assumption. In fact, some models are just shells into which complex parametric functions are inserted. The Black-Scholes equity option formula is currently used in practice with a breathtaking richness of the parameterization of the volatility "surfaces" describing different options including “skew” effects. The volatilities cannot be separated from the assumption of simple diffusion and the algorithms used.

.

Too Much Mathematical Rigor in Finance?

… following Feynman (ref: Feynman and Hibbs), it is difficult see the utility of a full-court press for rigor when financial models are only approximate, i.e. various assumptions behind the models are manifestly violated in the real world.

There is, moreover, a serious case against too much mathematical rigor in finance. Rigor can hide irrelevance. Rigor teaches us nothing new of practical importance. Rigor can be counterproductive because it makes the subject appear harder than it really is. The worst is that rigor gives a false sense of model validity.

The use of excessive rigor in finance parallels physics in the 1960’s for the mathematically rigorous axiomatic field theory. One paper (Gell-Mann et. al) put the situation in perspective: “In particular, the contribution of axiomatic field theory to calculations has been less than any pre-assigned positive number, however small”.

”

© 2009 Jan W. Dash. All rights reserved.

----------