Our Roadmap at Mithril Security
Our roadmap at Mithril Security to commoditize Confidential AI
Introduction
We have previously shared our mission at Mithril Security: democratize Confidential AI.
At Mithril Security, we believe Confidential Computing is a promising technology that has a lot of potential. It can help answer current needs for a secure, fast, and accessible solution to handle sensitive data.
Our adventure is only beginning, and we have recently open-sourced our core product, BlindAI, a secure AI deployment solution to serve models with confidentiality guarantees.
While our solution opens new venues with the deployment of state-of-the-art models such as BERT with privacy, we are aware that our project still has some way to go before being used in production. Once that stage is reached, our project will help cover the use cases we have identified: from confidential vocal assistants to healthcare analysis through document analysis, through vocal recognition.
We think it is vital to share our vision of the future and where we are headed both as an open-source project and as a startup.
The different milestones we will present are somewhat sequential but some can be pursued in parallel, depending on requests by our community and clients, and on the amount of manpower at our disposal.
Confidential AI deployment
BlindAI is our first step in our Confidential AI roadmap to democratize privacy-friendly AI solutions.
We want to provide a unified, simple, and secure solution to deploy models with privacy guarantees thanks to end-to-end protection.
We have two areas of work for BlindAI:
- Preprocessing/post-processing: we want to provide a unified interface to cover all the steps necessary for the deployment of models. While we provide a secure ONNX backend, we want to include the preprocessing and postprocessing steps that are often necessary, with a single interface.
One lead is to follow the work on ONNX extensions and to implement further operations in our Rust backend, Tract, for instance, to include Tokenizers seamlessly, without requiring to touch the Rust codebase. - Connectors: we currently only provide Python as a Client SDK. However, data owners could rely on other languages or platforms. For instance, if we want to secure the sending of vocal data from a mobile phone for speech-to-text, we will need to provide an Android / iOS client which will perform the remote attestation.
Therefore we need to provide different client implementations to cover those use cases, from SDK for mobile to Web solutions.
Our open-source approach intends to advance the state of Confidential AI as a whole. Enabling the community to support our project is key to the global development of such technologies. We therefore greatly welcome aid for the development of BlindAI.
Confidential training
While deployment is key and the last milestone to unlock value from AI, we are aware that training on confidential data is also a challenge. We have chosen to begin with Intel SGX as it was at our beginning in 2020, the most mature solution, and the one with the best tradeoff between speed and security.
CPUs can provide more than reasonable performances for inference, as Hugging Face showed for instance with millisecond latency for the BERT model.
However training is more complicated, but the real deal is coming: Nvidia will provide confidential GPUs with the coming H100! This means it will be possible to train large models on large datasets with security and privacy guarantees in a reasonable time.
We intend to proceed in two steps to tackle training:
- First, provide the ability to fine-tune models on a single enclave. For instance, by providing a Pytorch-compatible interface for people to send model architecture to an enclave, and allow data owners to share their data securely to the enclave.
- Then, explore federated learning with enclaves, data, and/or model-centric ways. The use of enclaves would protect the confidentiality of the data and model, while also providing proof of execution for transparency.
Diversification on Architectures and Clouds
We have chosen to start with Intel SGX as it is the most tested solution at the time of the writing of the article and provides a paradigm where with careful engineering we can provide the best guarantees of security thanks to the minimal TCB.
Our current focus with Intel SGX enables us to naturally onboard users that are currently on Azure, as Azure is the main Cloud provider with an offer with Intel SGX machines.
However, we want to cover other Confidential Computing solutions to answer our users' needs and cover:
- AMD SEV, which will enable us to increase our presence on GCP.
- Nitro enclaves, for the deployment on AWS.
Enterprise and Cloud offering
In addition to the products we mentioned above, we provide features to help our clients adopt and scale our solution for real deployment, including:
- Managed Cloud solution with a fully managed SaaS offering, for a drag and drop of models, and a PaaS offering to deploy BlindAI on your subscription
- Managed deployment on a Kubernetes cluster
- User and role access management
- Complex key management
- Metrics for deployment tracking in a confidential manner
There is still a long way before achieving all this, and we hope you will help us on our mission to commoditize Confidential AI!
If you want to support us, please star our GitHub repository, and come exchange with us on Discord!
