Blog

This is a transcription of our interview with Terisa Roberts, Director and Global Solution lead for Risk Modeling and Decisioning at SAS about the role of AI/ML in Risk Management.

You can watch the original video interview here or tune in to this episode on our Risk Management Show podcast here https://globalriskcommunity.libsyn.com/terisa or via iTunes, Spotify and other podcast apps by searching "Risk Management Show"

Boris: Welcome to our interview with Terisa Roberts. She is a Director and Global Solution lead for Risk Modeling and Decisioning at SAS. She has extensive experience in quantitative risk management, and advanced analytics. She has worked in a variety of industries, including financial services, telecommunications, energy, retail and government.

She advises banks and regulators around the world on best practices, topics in risk Modeling, Decisioning, and the responsible use of artificial intelligence and machine learning. She holds also at PhD in operations research and informatics and lives with her family in Sydney, Australia. Terisa, thank you for coming to our interview today.

Terisa: Thank you, Boris. It's my pleasure.

Boris: It’s my pleasure too. SAS has been an analytics leader for a long time. It has been named one of the top risk and compliance technology providers by Chartis for 15 consecutive years. It ranked fifth overall in the Global Chartis RiskTech100® 2021 report and won three industry solution categories where Model Risk Management being one of the three.

A few weeks back we had an interview with David Asermely about Model Risk Management and today we will continue more on advancement of AI/ML technologies in Risk Management. Terisa, could you perhaps explain us in a few sentences what you and your team at SAS have been up to these days?

Terisa: Sure. Boris, absolutely. So, SAS has always been a leader in providing a world-class analytical solutions for a wide range of business applications. These include the risk management, fraud detection and customer diligence as well. What is exciting for us at the moment is we've started to move and making our solutions available on our SAS Wire which is a cloud native platform.

So that opens up a whole lot of new opportunities for us to innovate.

Boris: The COVID-19 pandemic has massively disrupted our lives. How did it impact risk models? I believe that there is a Pre covid saying that “Essentially, all models are wrong, some are useful which now can be transformed “Essentially, all models are wrong, some are useless”

Terisa: Yes, most turned out to be useless. And that is just because the model inputs have moved outside of the calibration range of these models. The models have never seen the narrative of a pandemic play out. So we've seen lockdown measures in many countries that had impact on unemployment and on movement restrictions, et cetera. So we as individuals have been massively disrupted, but so have the business models of companies.

So we we've seen disruption at a global scale. And what we noticed on the risk management front is that not all the industries where impacted in the same way. So some were harder hit than, than others. And the recovery path is still uncertain of what will happen now. I just want to add to that, that we saw both the traditional statistical and econometric models breakdown, as well as the newer algorithms such as machine learning.

Both models have had trouble in coping. So financial institutions had to apply their own human judgment and make adjustments and management overlays with those in place for the models.

Boris: Based on your work with the financial and the regulators and the firms around the world, what are some of the kinds of AI/ML applications that are delivering tangible benefits?

Terisa: Sure. What we see with the use of AI and machine learning for risk management, it's really good at jobs that require repetition and that can be automated. So AI and machine learning is good at trolling through volumes and volumes of unstructured and structured data and pick up patterns that the human eye might miss. So possibly it's a less glamorous than what we see in the movies, but definitely it's starting to pay off and deliver tangible benefit in the areas of credit scoring.

So being able to offer credit and access to credit, to minority groups where perhaps the credit Bureau funds are not available and also in the area of a fraud detection and cybersecurity.

Boris: Interesting. And what did you learn specifically as a risk management practitioner? Because with new technologies come new risks, what are the main challenges that firms are facing in the use of these new technologies?

Terisa: This is a great question, Boris. So in the area of risk management, I think there's still a lot of caution in the industry. These are known risks that we associate with the use of these more complex and sophisticated algorithms, say they lack transparency. It’s not so easy to explain to the various stakeholders how the model works but there's also a great strides being made in addressing the explainability of all of these algorithms.

The other challenge that is often associated with AI and machine learning is that in may perpetuate and amplify a bias that might be present in your training data. So historically we, we might have some individual and societal biases, and that might be baked into, into the data that these algorithms are not smart enough to adjust for. So, we might be amplifying those biases with a user and we've seen this in the news.

We saw it with facial recognition systems and in the credit space and approvals of credit cards, some big technology companies gave the women a much lower credit limits because of the historical data. But by having said that if I may, a bias definitely is a concern there always to address it.

And once we've addressed it in these models, the AI and the machine learning models might even help us to make more consistent data-driven decisions. So definitely not in an insurmountable challenge.

Boris: Are existing model governance frameworks sufficient for AI/ML or there should be new ones developed and approved by different bodies?

Terisa: Excellent. Yes, we have people mentioning, how about AI validating AI? So can we use AI models to also monitor these? Of course the shelf life of some of the models might be shorter than our traditional models because of the sophistication in the patterns that they've detected and maybe the robustness. 

So, we need to up our levels of validation, to make room for AI and machine learning and its putting a lot more demand on validation teams.

Also in the areas of a feature engineering that needs to be validated as well as other hyper parameter tuning methods that has been employed. So putting a lot more demand on model governance and it's certainly not going away with the use of AI and machine learning.

Boris: So if I understand to be better prepared for new types of Risk, how can firms take more strategic approach to the use of this new technology and the risk management from your opinion?

Terisa: So what we we've seen happen is that the process that takes to put models, whether they are traditional or more innovative into production is still long, on average it's six to nine months for the financial services industry. So, for banks to take new innovation and Modeling seriously, they would need to rethink the model life cycle process, and look for efficiency gains perhaps in standardizing the data that goes into the models as well as looking at the deployment.

Ideally if you think about it, a traditional model might have a handful of input variables. If we look at an AI or machine learning model that can easily turn into hundreds and thousands of input variables. So if you have a manual process of deploying those models into your decision architectures, for example, that might not be scalable for you as the, the models increasingly grow in the number.

Boris: So I would like to ask you a personal opinion, what a risk managers should start doing in this field of a new technology of AI ML that they are not doing, or maybe also another way around, from your opinion, what they should stop doing that they are doing now.

Terisa: Yes. I think that there’s a hesitation to embrace new technologies. And I understand that because there's also a lot of hype around the use with computer vision and robots doing people's work, but that is not the case. That is more than the science fiction. So if we look at the science fact, there are areas where innovation can make a real difference, we see it in customer experience.

So by having these models to estimate income, you can gain efficiency and get much more accurate data. So, look at perhaps not replacing all your traditional models with machine learning. That's, not what I'm saying, but look at where the AI and, and the machine learning model can give you additional value of perhaps in the auxiliary function, in your data quality processes, as well as in the feature engineering process, which is typically in the model of development life cycle, also quite a manual process.

Boris: Just to summarize the major takeaways, if someone who is listening to this interview, would like to walk away with one or two major takeaways, what would it be from your point of view?

Terisa: I've alluded to it to an earlier, but when it comes to AI and machine learning, we need to rethink our architecture, the infrastructure behind the data management, the Modeling, as well as the deployment of the models and making sure that the infrastructure is future-proof to handle the sophistication of these models and look for areas where the manual processes can potentially be improved with automation, for those efficiency gains and scaling because the operational efficiencies or quite substantial,

Boris: From our perspective for myself, as I’m running Global Risk Community, a social network for risk managers over the world, what do you think, how can we contribute to a better understanding of this, a complex world of Risk Management?

Terisa: We see the, the complexity increasing, right. There are a lot more emerging risks, Climate risk, geopolitical risks and cetera. So as a community, as the front representative of the industry, regulators, risk management professionals, technology providers, if we collaborate with solutions that are practical and help us to operationalize more of the technologies, that it will make for a better world in a safer place, more equitable and fair outcomes as well from our Risk decisions.

Boris: All right. Thank you. This is what I wanted to ask you, and maybe you, if you have some additional thoughts on something that I didn't ask you.

Terisa: No, it's been a great conversation. I love talking about a new innovation in Risk, so hopefully we can continue the conversation at some someday.

Boris: All right. Thank you very much. I wish you and your company a great success in implementing these new technologies in the Risk Management space.

Terisa: Thank you. It's been a pleasure.