We are excited to introduce BlindBox, our latest open-source solution designed to enhance SaaS deployment security. Our tooling enables developers to wrap any Docker image with isolation layers and deploy them inside confidential containers to offer complete end-to-end protection, verifiability, and control to data owners. With this article, we invite you to learn more about BlindBox and how it can improve the security of your SaaS deployment.

Context

Cloud computing has been one of the main catalysts of tech innovation over the past decade, powering thousands of successful startups with cutting-edge solutions. It has fueled the success of countless startups that range from OpenAI's ChatGPT, (running on Azure's powerful GPU infrastructure), to Snowflake's cutting-edge data warehousing and analytics platform.

However, using SaaS can lead to high privacy and security challenges due to data exposure when sending it to third-party providers. Until now, software vendors had to choose between sacrificing agility with on-premises deployment or providing a scalable SaaS solution that risked losing customers unwilling to send data outside their trusted perimeter.

Mithril Security has been on a mission to solve those privacy and security challenges for privacy-focused SaaS customers, starting with securing AI workloads.

Our first product, BlindAI, is an open-source solution that deploys AI models with privacy protection using hardware-based Trusted Execution Environments (TEEs) with application-based secure enclaves, one of the most secure hardware-based solutions available. We implemented a lightweight AI inference solution in Rust inside the enclave, which Quarkslab, an independent security lab, successfully audited.

However, we felt BlindAI would not be enough because it is:

• Hard to develop with secure enclave constraints
• Hard to maintain
• Restricted to AI inferences as we only support ONNX files, which makes pre/post-processing complicated.

We believe that adopting a security solution is based on three factors: ease of use, speed, and security. While BlindAI answers very well the security requirements, it was too limiting for vendors and not fast enough.

So, we built BlindBox, a flexible, fast, and secure tooling to deploy SaaS apps inside secure confidential containers and recreated the isolation of on-premises air-gapped environments. Confidential containers are a more flexible solution to deploy applications, creating Trusted Execution Environments with a smaller Trusted Computing Base that is powered by the new generation of confidential hardware, AMD SEV-SNP.

Customers can consume an external SaaS and have technical control over who is authorized to see data and where data can go, as they are the only ones with the keys. Even SaaS vendors cannot see customers' data and thus cannot expose it even if compromised.

With BlindBox, software vendors will be able to leverage SaaS to provide the best customer experience (scalability, easy onboarding, latest features) while ensuring the same level of privacy and security as on-premises.

Introducing BlindBox

BlindBox is an open-source tooling solution that enables developers to wrap their apps in isolation layers enforced by secure hardware and verifiable by remote parties.

While our initial product BlindAI, could only take ONNX files, BlindBox can now take arbitrary Docker images and deploy them inside confidential containers.

By adding network isolation to the Docker image and deploying it in a confidential container, we can effectively create an air-gapped environment. We add an authentication layer and ensure that both authentication and network isolation are enforced through remote attestation. This way, data owners can verify that the application they are communicating with:

• does not expose data to the SaaS provider
• cannot exfiltrate data outside of the secure environment.
  

We create a virtual extension of data owners' infrastructure. This is what we call a “Virtual Private SaaS.” which is a SaaS deployed inside isolated and verifiable confidential containers. Virtual Private SaaS reproduces an air-gapped environment similar to on-premises deployment, thanks to the end-to-end protection, verifiability, and control confidential containers provide.

We leverage confidential containers on Azure Container Instances (ACI) to provision the low-level layers, from sourcing the right secure hardware to providing the software layer to run Docker images. This is done by provisioning containers leveraging AMD SEV-SNP technology.

BlindBox then adds the isolation layer to existing software vendors' Docker images and deploys it on ACI.

Confidential AI: Customer stories

While BlindBox’s final mission is to help any SaaS vendor provide the privacy guarantees to deploy to demanding customers, the first step of our journey is to continue focusing on confidential AI, with a special focus on large language models (LLMs).

BlindBox's ultimate objective is to empower every SaaS vendor with the privacy assurances needed to win the trust of their most demanding clients. As we embark on this journey, our first step is to relentlessly pursue our confidential AI vision with an unwavering focus on advancing LLM technology.

We have, therefore, collaborated with several customers to deploy privacy-first AI-based applications. You can find below stories from our first partners.

Digital Forensics with Avian

"Thanks to our collaboration with Mithril Security, we've created cutting-edge Digital Forensics tools for Avian Cloud.

BlindBox enables us to use LLMs, speeding up investigations while data customer privacy, as not even Avian can see customers' private documents in our SaaS solution. With the benefits of on-prem deployment and the flexibility of the cloud, we're proud to offer the most advanced platform for Digital Forensics."

- CEO of Avian

Medical voice notes transcription for the NHS

“Mithril Security is the solution I have been looking for to answer the privacy issues to provide AI-assisted tools to physicians at the NHS.

Physicians at the NHS often have to take many notes and spend time doing administrative tasks that distance them from patients.

We have been wanting to use AI to transcribe medical voice notes and turn the raw text into filled forms to save physicians’ time.

However, ensuring data privacy when sending patient data to outside AI solutions is a real issue that can be blocked for us.

By using BlindBox, we could get the best of both worlds: leverage state-of-the-art AI solutions from third-party vendors to create value for physiciansthe without risking the exposure of patient data”.

- Director of Tramscribe Ltd

Get started with confidential AI

You can get started with Confidential AI easily with our Quick tour that shows how to deploy the OpenAI Whisper model with BlindBox for confidential speech-to-text! This is what we have been using to help transcribe and analyze medical voice notes for the NHS.

The steps to get onboarded with BlindBox are fairly simple:

1. Prepare your Docker image
2. Use our CLI tool to package your image with isolation layers
3. Deploy your modified image using our CLI tool
4. Query the remote AI model with privacy guarantees using our client SDK
   

Learn more about confidential computing

The technology we use behind the scenes is called confidential computing, which uses hardware-based solutions to secure the analysis of sensitive data. You can learn more about it in our tutorial or in our series Confidential Computing Explained, which teaches how Trusted Execution Environments work for developers by showing how to code a key management system inside a secure enclave.

A key collaboration for this journey to democratize confidential AI has been with Microsoft. As a leader in AI and confidential computing, this collaboration was natural to us. We are pleased to advance our collaboration using confidential containers on Microsoft Azure Container Instances (ACI) as our deployment solution for BlindBox. Vikas Bhatia, Head of Product at Azure confidential computing, says:

“Mithril Security’s BlindBox is a product that aims to reconcile data privacy protection within a software-as-a-service (SaaS) offering. Their first focus on confidential AI is promising, as large language models are changing entire industries. Confidential AI can speed up LLM adoption by sensitive industries like healthcare, finance, or government where many customers seek privacy assurances on their data.”

Next steps

Our goal for the future is to democratize Confidential AI by helping AI vendors deploy to sensitive industries.

We will continue democratizing Confidential AI for the future, with lots of integration with LLMs frameworks such as Langchain for Confidential Chatbots or llama_index for Zero Trust Search, so stay tuned and register for our newsletter!

If you are interested, please try our Quick tour and star our GitHub! ⭐
If you face issues deploying your SaaS to privacy-demanding customers, do not hesitate to book a demo with us!

Want to turn your SaaS into a zero-trust solution?