Model Risk and Validation - Fund managers and traders rely on models for many purposes, including valuing illiquid investments, measuring risk sensitivities for hedging, managing market and credit exposure limits and risk reporting for efficient portfolio and firm level risk management.
Fund managers and traders rely on models for many purposes, including valuing illiquid investments, measuring risk sensitivities for hedging, managing market and credit exposure limits and risk reporting for efficient portfolio and firm level risk management. Irrespective of the purpose, firms should ideally validate their internal models to avoid financial losses.
Financial loss can come from the use of an inappropriate model and also from operational flaws in using a sound model. The losses can be quite immediate in a pricing and valuation context but, in case of hedging, it can accumulate to a large amount over a period of time. In this article we define “model risk” and its variants and subsequently discuss appropriate steps and measures that can be taken to address and mitigate it.
Defining “Model Risk”
For most financial securities (except exchange traded instruments and “plain vanilla” liquid OTC positions) we cannot observe the prices directly in the market. A model which is a mathematical derivation is devised to estimate the price based on the underlying asset or risk factor dynamics. This process is called “mark to model” and is exposed to estimation errors due to mathematical formulae involved and the subjective components of choosing liquid instruments in calibration of model parameters.
We can define the model risk as an estimation error where “mark to model” results in a price(s) significantly different from the price at which the asset has been recently transacted in the market1. Another way to interpret model risk is the possibility of significant underperformance of a trading strategy where hedging is performed based on the risk sensitivities of the implemented model.
In general model risk emanates from applying a model which is not appropriate to solve the pricing and hedging problem at hand or from operational flaws in implementing or using an otherwise robust model. Model risk not only exists for exotic derivatives but also for securities with high illiquidity in certain market conditions. Therefore, funds trading in securities like leveraged loans, high yield bonds, convertible bonds, mortgage backed securities or any exotic derivatives need to be aware of the types of model risk they can face and take measures to address them. At the same time model risk is not limited to traditional asset pricing and risk models but it can exist in quantitative strategy models too.
For example, the strategy does not perform as expected in all market regimes or again there has been some mistake in implementing or using otherwise sound models. Recently we have seen a case where an investment manager, AXA Rosenberg, settled with SEC and investors for $ 242 million2 due to a material error in its model.
Classification of “Model Risk”
It is worth highlighting types of model risk that have been best categorized by Derman3 in a Goldman Sachs internal paper.
1.Inapplicability of modeling: The mathematical modeling framework being deployed is not applicable or relevant for particular security or asset class valuation.
2.Incorrect model: Model is incorrect when certain assumptions behind the model are not appropriate. For example, not having sufficient factors that drive valuation and risk or assuming a factor to be deterministic when it should be stochastic. An Incorrect model can also result from assuming incorrect distribution of variables (e.g. normal instead of lognormal) or assuming wrong correlation among market factors. Estimate of volatility or correlation based on bad data is also one of the most common mistakes.
3.Correct model, incorrect solutions: Even when the model is correct, making a technical mistake in solving the model.
4.Correct model, inappropriate use: Using wrong model inputs. For example, choosing very few simulations paths for valuing a complex derivative in a Monte Carlo simulation.
5.Badly approximated solutions: Error resulting from an approximation in the numerical method applied to solve the model.
6.Software and hardware bugs: Models are hugely sophisticated programs which interact with market databases, user interfaces and multiple price feeds. This category includes existence of any mistake in programming logic in any of the components.
Disclaimer: Dime-Co.Com is an online information article and video article network. All articles, video articles, comments, and other features herein are for informational purposes only and are provided "as is" without warranties, representations or guarantees of any kind. The views and opinions expressed in an article, comments, links or blogs are the author's own, and not necessarily those of dime-co.com's owners. For full disclaimer, please read our TOS.