Errors in financial models that banks use on a daily basis could lead to tremendous financial and non-financial losses. It is crucial for banks to understand how they could minimize and manage model risk effectively. In addition, the OCC and the Federal Reserve have recently released new guidelines on model risk management, which will significantly modify their existing model risk management practices.
Vilen Abramov, Vice President, Model Risk Control at KeyBank answered a series of questions written by GFMI before the forthcoming 2nd Edition Model Risk Conference, June 10-12, 2013 in New York, NY. Mr. Abramov shares his thoughts below on the management of risks associated with VAR.
What are the key processes and techniques for ensuring model risk is minimized in VaR models?
Vilen Abramov: KeyBank has established three lines of defense for the model risk control. In case of VaR models, the first line of defense is the Market Risk Management (MRM) group. As the first line of defense, MRM has established controls for all stages of the VaR production process in the following manner.
- inputs stage controls
(i) positional data reconciliation
(ii) terms and conditions mapping
(iii) market data integrity testing
- processing stage controls
(i) risk factors set completeness and granularity review
(ii) scenario definitions review
(iii) calibration procedures review
(iv) benchmarking pricing models
(v) documenting modeling assumptions/limitations/simplifications
- output stage
(i) backtesting general VaR (GVaR) and stressed VaR (SVaR)
(ii) p-value testing for GVaR and SVaR
(iii) stress testing and sensitivity testing
MRM has established documentation standards which are in line with OCC 2000-16 and FRB SR 2011-07 guidance. It covers three major modeling validation stages: evaluation of conceptual soundness, evaluation of ongoing monitoring and evaluation of outcome analysis.
The second line of defense is the Quantitative Risk Analysis (QRA) group. QRA independently validates VaR models on an annual basis. This role is responsible for reviewing the tests created by the MRM, for creating additional tests to their own satisfaction, and for making the final assertion that test coverage and depth is sufficient.
The third line of defense is the Risk Review Group (RRG). RRG reviews the first and second lines of defense validations and ensures that the model validation standards are met. This role may also run additional tests if needed.
What are the latest techniques and approaches for quantifying model risk?
VA: At KeyBank, MRM group is acting as an independent model validator for Treasury and Capital Markets areas. MRM has established performance metrics for all models and urged lines of business (LOBs) to start collecting the model performance data. This will allow MRM to quantify potential losses caused by the models' underperformance.
As the first line of defense, MRM has established VaR performance metrics as well. In addition to the VaR backtesting and p-value testing, MRM uses the following metrics.
- VaR netting effect
VaR is neither sub-additive nor super-additive measure. In the case of super-additivity, using marginal VaRs for risk weighted assets (RWA) calculations may lead to a lower capital. On the other hand, in case of sub-additivity, using marginal VaRs for RWA calculations may lead to lower number of exceptions, and as a result lower multiplication factor and lower capital. In order to cure the potential underestimation of the regulatory capital, MRM has quantified the impact of the marginal VaR approach by comparing it to the netted VaR approach.
- VaR scaling effect
It is well know, that the “square root of time” scaling (e.g. when converting 1-day VaR into 10-day VaR) may lead to VaR underestimation. MRM has quantified the impact of the scaled VaR approach by running real 10-day VaR and comparing it to the scaled version.
- SVaR time period selection
MRM has established the SVaR/GVaR ratio metric to measure appropriateness of the time period selected for the SVaR calculations.
- VaR regime shifting
MRM has established “non-equally weighted VaR/equally weighted VaR” ratio to measure models’ sensitivities to the changes in the market conditions.
What is the role of internal audit in managing model risk, for example, in VaR models?
VA: As it was stated earlier, RRG is KeyBank’s internal audit team which manages model risk. Its role is to ensure that the development process is followed (including sign off for model validation and for documentation), to ensure independence of VaR modelers from the affected LOBs and to ensure compliance with the regulatory and internal guidance and policies. As a trusted advisor, RRG makes recommendations for improvement, monitors remediation efforts and helps to ensure that adequate risk management practices are followed. The results of these activities are provided to key stakeholders, including the Board of Directors and associated committees, executive and line of business management and regulators.
How could you ensure model risk management is an integral part of the wider enterprise risk management programme?
VA: KeyBank’s model risk control function is residing within Risk Management department. The model risk management responsibilities are handled by the three Risk Management subgroups: Quantitative Risk Analysis, Market Risk Management, and Strategic Analytics. These three subgroups along with RRG, Credit Portfolio Management (CPM), and Treasury constitute Model Risk Management Committee (MRMC). The MRMC provides governance and oversight of KeyBank’s model risk management. The committee operates as a subcommittee of the Enterprise Risk Management Committee (ERMC).
You have attended our first Model Risk conference, what do you think you would gain by attending the second edition?
VA: It is a great opportunity to learn the best practices and listen to the new ideas. Having regulators and authors of the model risk management guidance share their ideas, provides invaluable insight into the field. Presentations provide invaluable information. Networking and building connections is very important in such a dynamic and fast growing field like model risk management.
Since joining KeyBank in 2007, Vilen Abramov has risen steadily through the ranks to become vice president of modeling and model validation for market risk management, treasury and capital markets. His expertise comes from first-hand experience in market risk management, internal audit, and model validation areas. Vilen holds a Ph.D. in applied mathematics and PRM certification. KeyBank, headquartered in Cleveland and owned by KeyCorp, is the 25th largest bank in the United States with more than $86 bn in total assets.
For more information please contact Michele Westergaard, Senior Marketing Manager, Media & PR, marcus evans at 312-540-3000 ext. 6625 or Michele@global-fmi.com.
Comments