Financial services executives view artificial intelligence (AI) with a powerful mix of excitement and concern. With new proofs of concept, innovation labs and investments in technology appearing every day, financial institutions – including insurers - are eagerly exploring AI to improve business decisions, customer experiences and risk management outcomes. While AI promises great opportunities to work faster and smarter, it faces substantial scepticism among stakeholders, including regulators — due to the “black box” nature of some techniques and the speed with which the models are developed and changed. The spread of AI through financial services will be severely hampered unless users overcome this scepticism by adopting a systematic approach to managing AI models through their life cycle, from cradle to use to retirement.
AI is such a broad category that it defies simple description, but it typically refers to a suite of modelling techniques that bring together some combination of the following: huge data sets, non-traditional (i.e., including unstructured and changing) data, demonstrating complex relationships between variables sometimes result in opaque (“black box”) models, and models with rapidly time-varying structures. As AI provides previously unknown insights, insurers are implementing AI models in order to increase revenue or reduce cost through better and faster decision-making. Customer segmentation, fraud detection, price optimization, compliance monitoring, and loss forecasting are only a few examples of areas where financial institutions have built models using a range of approaches such as clustering algorithms, deep neural networks, and sentiment analysis.
Over the past few decades, model management frameworks were designed around traditional models, and today, AI models present challenges for those existing frameworks. For example, current model management practices generally rely on regular model validations, scheduled or based on “material changes,” which work well for traditional approaches as they are generally updated annually or bi-annually. However, how would a model that is updated daily or even in real-time fit in this validation framework? Data is another example. Within the traditional framework, model development generally begins once a cleaned up and curated dataset (with outliers, missing and invalid data points accounted for) is ready for review by model risk management. Within the AI paradigm, datasets can be so large that such a degree of curation may not be feasible or necessary. How will model risk management review and provide a validation outcome for such datasets?
These discussion points are starting to arise and cause concern in institutions experimenting with AI. In some cases, it is leading to AI models not being put into production. Financial institutions have three groups of stakeholders voicing their worries: the Model Owner, the Model Reviewer and the Model User. We illustrated the concerns of these three groups below.
These concerns can have a tangible negative impact as investments in AI can be lost or cost the firm more time and resources while the model is forced through the traditional framework. Institutions planning to invest in and implement advanced AI capabilities need to start adjusting their model management framework, and address the concerns of the three groups of stakeholders. It’s a significant effort with multiple layers to get to a state where model management frameworks are modernized to fully accommodate AI and other advanced analytics. Rather than define a multi-year roadmap to get to this state, we have laid out the seven paths that financial institutions should start exploring in order to start moving in the right direction.
As the above paths are explored, financial institutions need to keep communication channels open with the regulators to demonstrate plans and progress toward building a robust model management framework which accommodates AI and is resilient to future innovation. This transparency with the regulators will play an important role in identifying and addressing concerns from a regulatory perspective.
Despite its critical importance, establishing the model management infrastructure that enables the successful adoption of AI has not been getting as much attention as the actual development of AI. Without a framework to manage and govern AI, organizations will not be able to reap its full benefits. Therefore, financial institutions must begin their journey now. Those that move first in this critical transition will establish a long term strategic advantage as their potential to explore and take advantage of the benefits of AI will not be limited by model management, governance and other practical obstacles.
This was adapted from the Oliver Wyman report Managing Next Generation Artificial Intelligence in Banking by Jeffrey Brown, Tammi Ling, Ege Gurdeniz.