Confidential AI
Confidential Computing and AI or Confidential AI
In my recent discussions with customers across industries, I sensed a growing interest in Confidential Computing and Confidential AI based on the design patterns implemented in financial services and healthcare industries. Ensuring trustworthiness in AI applications is critical to reap its benefits. Reliability is a critical aspect of trustworthy AI. Improving reliability in machine learning models requires diversity in data sets used for training. So, peer institutions in financial services and healthcare are collaborating to share data and AI models for improving accuracy to improve prediction in use cases such as fraud detection and medical diagnosis. Hence, CISO’s and CIOs at many of these institutions explored privacy preserving machine learning techniques like Confidential AI and Confidential Federated Learning. Before I delve into these topics, let us see what confidential computing (CC) is and how this technology is important for AI.
Data protection today follows two principles — data at rest and data in transit. Our data in our laptop hard drives, servers in the data centers or the cloud are encrypted. Protocols such as HTTPS and TLS protect data in transit — that is data moving from our desktops and mobile devices to websites or through the network are encrypted. But what about data protection when your code is running in the servers on your preferred cloud provider? When your code is running on your cloud VMs, the data is in the server’s working memory. Data in memory is generally not encrypted for the application code to access. Attackers can exploit this vulnerability and extract the memory content. Till recently, security for “data in memory” was not enforced by regulatory bodies. But regulatory agencies, particularly in financial services like the European Banking Authority are providing guidance to banks, insurers and other entities on the strict controls for data in memory. Protecting data in memory is critical only to prevent data breaches but also to meet compliance requirements for GDPR, HIPAA and PCI DSS.
CC provides this missing piece. This is a technology that protects data in memory and in use through a hardware based Trusted execution environment (TEE). A TEE is a secure “enclave” within a CPU. TEEs are secured using embedded encryption keys accessible only to authorized code. During computation or processing, the application code can instruct the TEE to decrypt the data. When decrypted, the data is not accessible to the operating system or the employees of the cloud provider but only to the authorized. With CC, data is protected in every stage of the life cycle — at rest, in transit and during use. Intel and AMD have specialized CC CPU technology available for cloud providers.
CC is not just for securing data but also your machine learning models. Companies are not only risking their intellectual Property by training these models in non-confidential environments on the Cloud but millions of dollars of investment that is required to develop these custom models.
Confidential AI is the intersection of AI and CC. This practice uses the principles of CC principles to protect data used to train machine learning models and the model themselves including their parameters, model weights etc. Although all cloud providers offer Confidential VMs and Containers, Microsoft Azure is generally seen as a leader today in providing offerings to secure data and AI workloads. Azure CC offers services for CPU based AI workloads and Containerized AI workloads. Microsoft and NVIDIA are collaborating to bring CC to NVIDIA GPUs. At the time of this writing, Azure Confidential GPU VMs are in limited preview.
In the financial services, collaborative research is important for fraud detection and anti-money laundering investigation and risk assessment. This approach called multi-party computation enables joint analytics on encrypted customer data. Financial institutions have built machine learning models and rely on their data to discover fraud. But bad actors can orchestrate fraud across many institutions. Multi-party computation (MPC) platforms enable data sharing between financial institutions to detect behavior patterns and red flags across the systems. These platforms rely on CC technology for data security and privacy. Accenture Labs’ MPC solution for the insurance industry can be found here.
The financial services and healthcare industries also implement an approach called federated learning (FL). These industries also share models to overcome data imbalance. Simply put, FL is a method to train AI models without anyone seeing or touching your data. For improving fraud predictions for example, multiple banks can download pre-trained foundation model, train it on their private data, then summarize and encrypt the model’s parameters, weights and send back to a model aggregator on the cloud, decrypt, average and integrate into the centralized (or aggregated) model. This iteration and the collaborative learning continue until the model is fully trained or the model converges to an acceptable loss value. But the centralized model is vulnerable to attacks including model poisoning, data poisoning and inference attacks. Confidential federated learning (CFL) — combining both CC and FL technologies is a new paradigm that can prevent attacks on FL and encourage the adoption of FL. With TEEs, model weights for example, are exposed only to the attested client code. On the aggregator side, all access can be limited by hosting the trained in a TEE. CFL can be implanted using NVIDIA Flare, a commonly used FL Framework and deployed on Azure’s confidential containers or confidential VMs.
In conclusion, confidential computing can activate several use cases within an enterprise while preserving the principles of security, privacy, accountability, transparency and fairness. Reliability in AI applications can be improved through collaboration, data and model sharing. Use cases that require multi-party computation and federated learning approaches can now be designed and deployed in confidence with confidential computing.
