In this week's blog post, we're sharing insights on our interview with David Asermely. David is a global Model Risk Management Lead at SAS driving strategic conversations with global institutions and influencing the SAS model risk management solution roadmap. He is passionate about translating data into actionable intelligence, and he focuses on combining the best technologies and design principles to improve modelling efficiency and quality. Based on that, we invited David Asermely to talk about the importance of the MRM discipline, as well as some of the current challenges and threats.
Role of Model Risk Management
The main role of Model risk management it's to identify models that should be replaced, either because they are hurting your business or they are not providing the value that they should. SAS has also been working on a white paper that is talking around the importance of model risk management and what it brings an organization. One of the analogies developed in that paper is that your models are your digital workforce and you should make sure that your workforce is of the highest quality that is performing tasks as appropriate. This means that a Model risk manager’s job is to independently review the models and identify ones that should not be used.
Using robust model risk management and cutting edge analytic tools also creates a big competitive advantage. If your organization has a better overall model of quality that is removing models that are performing poorly, you are going to have a competitive advantage over an organization that does not have models that are performing. From a competitive advantage perspective, model risk management brings to an organization an additional level of certainty to the model, a modeling process that drives value up and down the Model operations perspective and also to the business perspective.
Despite its large use and competitive advantage, most of the Model risks have been focused on the larger banking and insurance firms, mainly in North America and Europe rather than other industries. This mainly comes from the fact that there have been a number of clear regulations in the financial markets, over the last couple of decades. These regulations that are continued to be enforced at these organizations on understanding where your models are, how they're performing, having that independent review associated with the model to bring that additional rigor and perspective to the modeling process.
The level of sophistication continues to rise in the world of model risk management, especially as machine learning models become more prevalent, things need to be done more efficiently yet still provide that critical information to these organizations. As financial markets have been the pioneers, most of the best practices have been developed there as well. And now we're seeing other industries wanting to take a look at what those best practices are and seeing how they can be implemented, so they increase the quality of analytics that they're using within their organization.
Risks Involving Model Risk Management
Pure reliance on models can bring some risks to the table, which can easily be avoided if proper implementation of MRM was done. David mentions that in their white papers they touch on many of these risks with historic examples. It is very important to understand that, one of the tenets of model risk management is understanding what could go wrong with that model, what is the absolute risk that could be associated with a poor model risk management. One of the things that is required in the domain of model risk management is to stress a model in various ways, to look at different data, extreme conditions on the different features of a model. By doing this type of a stress test, it often flushes out the potential damage that a model can cause within an organization. The subprime crisis is a very good example of this claim.
From a model risk management perspective, one of the most important aspects of the whole process of governing a model is understanding its potential. You have to analyse whether it is financial, or reputational- or if it's a combination of both. You have to consider if there is a possibility that a model can have you on the front page of the Wall Street Journal because your Model has a gender bias. These risks are only exacerbated with machine learning in which there's a capability of taking in a lot of other data that maybe historically has not been brought in. Another aspect is knowing the conditions in which a model will work properly. For example, if a model's performance starts to degrade quickly with interest rates getting closer to zero, then that's something that should be understood. And as those market conditions move that way, the model should be flagged, highlighted, and possibly decommissioned and replaced.
Introducing and Structuring MRM to Model Governance
MRM offers a lot to enable firms to manage and orchestrate introducing a structured approach to Model Governance. One advantage is that leading companies such as SAS have developed their solutions to work within your existing ecosystem as well as being future-proof. The capability to be a single repository that can govern your entire set of models in one location is quite important. The other thing MRM offers is the ability for the entire Model life cycle to be seamlessly connected to those tools. One example is, ensuring that models that are active and being used in production are actually approved in the Model Risk system and the answer is by connecting that system to your Model Ops in a way that requires that MRM sign-off before a model is moved into production and used.
By automating as much of the Model Risk system as possible and using machine learning, you also can boost performance of the team, rather than hiring more analysts. If the MRM system is guided to or coated a way to show where the data is coming from and provides usable data for your model risk management team, there is quite a lot of benefit in introducing and structuring MRM to your governance. Another area to focus on is figuring where your Model risk data is located. In most cases it is located in a database or spreadsheet accessible only by the team working on that data but not the entirety of the organisation. With development of tools allowing you to easily extract the most critical data can also help the rest of the organisation to be aware of what is happening within that field.
Creating a common MRM language that can be understood across the organization that's focused on the wellness of the Model and then making that information available via APIs can help bring MRM to the forefront within the whole modeling Community in a way that's constructive, in a way that will improve the overall usage and quality of a given model and will alert in real-time the users of the model of potential problems. The real benefit of investing in Model risk management is providing that type of information as you go forward and a lot of this can be automated. Having the capability where if you have models that are making automatic decisions for you, loan applications, for example, having the kind of capability in real time to adjust a model or turn a model off if needed or require human review in a way that prevents losses from racking up.
Benefits of Automated Model Risk Management
A big number of organizations that use models look at Model Risk as a cost. Instead however, this should be considered as a business expense, or cost of doing business.
Models bring automation, consistency to the organization and it provides information that allows them to better compete. Automated model risk management helps in that process and provides data to utilize those models in a more effective way. Automated MRM can actually allow you to save across an entire Model life cycle.
For many aspects of the modelling or roles involve modelling such as a model developer or a validator, it will definitely involve documenting the process or the data. And in most of these cases, if you ask a developer or a validator, if you ask what's the task that they hate the most, they will probably say it's documentation. There are other areas where these highly skilled, expensive members of your team are required to do a lot of boring, repeatable, mundane pieces of steps.
Having a Model Risk Management system allows you to take a look at those and reduce the manual effort and automate some of the documentation associated with it.You're always going to need that expert in this process, but, Model Risk Management allows the expert to focus on providing the expertise to the process, not doing some of the mundane components that typically are being asked in.
It is a similar thing for the Audit teams, as automating can also cut a lot of loss-time for these people. Whether this auditor may be internal or a highly priced consultant, often the first thing that has to happen is identifying the model's history. They will need all the documents and data they can get, which typically requires emailing several individuals who are not present at the office at the time.This means a lot of wasted time to collect all the information where that time can actually be spent for something else if those were readily available at hand. Having an MRM system where the history and the model is complete as a documentation and available to the auditor would both benefit the company and the auditor, make the process both time and cost efficient. For organizations that do this well, there is a value that is brought across an organization.
The last example is having a report that shows you how well you're doing Model Risk. This is an area where many organizations have senior people spending a number of hours trying to create a report where if you could have that automated and available at any time, it provides tremendous value. Overall, although at the start the implementation of MRM and the automation might seem costly and time consuming, in the long run it very well compensates for the time and money spent - especially if the right models and tools are chosen for the needs of the organisation.
Closing Words
For now, this sums up the key points of our interview. As the Global Risk Community team, we once again thank David Asermely for his insight on Model Risk Management discipline and topics around it. More information about this topic is available in our original interview, which is accessible here.
#risk #automation #modelling #management #mrm #governance
Comments