Canonical recently announced the release of Ubuntu 24.04 LTS, codenamed “Noble Numbat”. This update underscores Ubuntu’s ongoing commitment to enhancing performance and security, focusing on optimizing developer productivity. The latest version features an optimized Linux kernel 6.8 and significant system management upgrades as detailed in the release notes. In this blog post, we highlight the key features and improvements that Ubuntu 24.04 LTS brings to the table, specifically tailored for users of Microsoft/Azure.
Unified marketplace offering
Ubuntu 24.04 LTS introduces a consolidated Azure Marketplace experience. Easily find the official Ubuntu images created by Canonical and endorsed by Microsoft for Azure, all under a single offering: ubuntu-24_04-lts. This simplification aids your search and selection process, helping you choose the right image for your needs and ensuring optimal compatibility and performance. [Explore the Ubuntu 24.04 images on the Azure Marketplace].
Optimized for Azure
Ubuntu 24.04 LTS is finely tuned to enhance performance on Azure infrastructure, ensuring that the Ubuntu images are fully compatible and support the latest cloud features as they are released. This optimization boosts system efficiency, speed, and reliability. Integration with Azure Guest Patching and the Update Management Center facilitates streamlined and continuous system updates, thereby reinforcing the overall security and stability of Ubuntu deployments.
Enhanced developer toolchains
.NET 8 is fully compatible with Ubuntu 24.04 LTS from launch, being directly available through the official Ubuntu feeds. This synchronization with the .NET release cadence ensures developers have immediate access to the latest features and updates. Additionally, .NET 8 introduces streamlined package management and new Ubuntu container images, boosting development flexibility and deployment efficiency. (Read more in this Microsoft’s blog post).
The commitment to developer productivity also extends to other popular programming languages, including TCK-certified Java versions and the latest Rust toolchains, enhancing support and smoothing the development experience.
Confidential Computing
Ubuntu continues to lead in confidential computing with support for Confidential VMs, including capabilities for confidential AI. This is facilitated by utilizing advanced hardware security extensions such as AMD’s 4th Gen EPYC processors with SEV-SNP and NVIDIA H100 Tensor Core GPUs. These features help safeguard data at runtime from system vulnerabilities and unauthorized access, making them particularly suitable for AI training and data inference involving sensitive information.
Windows Subsystem for Linux (WSL)
Ubuntu 24.04 LTS enhances its WSL integration using the same installer technology as Ubuntu Server. This update includes support for cloud-init, standardizing developer environments across installations and ensuring consistent and streamlined workflows.
Wrapping up
As we explore the capabilities of Ubuntu 24.04 LTS, Microsoft/Azure users will experience an integration that is tailored to current technological needs and equipped for upcoming developments. This version is supported for up to 12 years, providing a stable and reliable foundation that enterprises and developers can rely on for long-term projects and innovation.
This blog post explores the technical and strategic benefits of deploying open-source AI models on Ubuntu. We’ll highlight why it makes sense to use Ubuntu with open-source AI models, and outline the deployment process on Azure.
Authored by Gauthier Jolly, Software Engineer, CPC, and Jehudi Castro-Sierra, Public Cloud Alliance Director, both from Canonical.
Why Ubuntu for Open-Source AI?
Open Philosophy: Ubuntu’s open-source nature aligns seamlessly with the principles of open-source AI models, fostering collaboration and accessibility.
Seamless Integration: Deploying open-source AI is smooth on Ubuntu, thanks to its robust support for AI libraries and tools.
Community: Ubuntu’s large community provides valuable resources and knowledge-sharing for AI development.
The Role of Ubuntu Pro
Ubuntu Pro elevates the security and compliance aspects of deploying AI models, offering extended security maintenance, comprehensive patching, and automated compliance features that are vital for enterprise-grade applications. Its integration with Confidential VMs on Azure enhances the protection of sensitive data and model integrity, making it an indispensable tool for tasks requiring stringent security measures like ML training, inference, and confidential multi-party data analytics.
Why use the public cloud for deploying AI models?
Using a public cloud like Azure gives straightforward access to powerful GPUs and Confidential Compute capabilities, essential for intensive AI tasks. These features significantly reduce the time and complexity involved in setting up and running AI models, without compromising on security and privacy. Although some may opt for on-prem deployment due to specific requirements, Azure’s scalable and secure environment offers a compelling argument for cloud-based deployments.
Provisioning and Configuration
We are going to explore using open models on Azure by creating an instance with Ubuntu, installing NVIDIA drivers for GPU support, and setting up Ollama for running the models. The process is technical, involving CLI commands for creating the resource group, VM, and configuring NVIDIA drivers. Ollama, the chosen tool for running models like Mixtral, is best installed using Snap for a hassle-free experience, encapsulating dependencies and simplifying updates.
Provision an Azure VM
Begin by creating a resource group and then a VM with the Ubuntu image using the Azure CLI.
az group create --location westus --resource-group ml-workload
az vm create \
--resource-group ml-workload \
--name jammy \
--image Ubuntu2204 \
--generate-ssh-keys \
--size Standard_NC4as_T4_v3 \
--admin-username ubuntu --license-type UBUNTU_PRO
Note the publicIpAddress from the output – you’ll need it to SSH into the VM.
Install Nvidia Drivers (GPU Support)
For GPU capabilities, install NVIDIA drivers using Ubuntu’s package management system. Restart the system after installation.
Important: Standard NVIDIA drivers don’t support vGPUs (fractional GPUs). See instructions on the Azure site for installing GRID drivers, which might involve building an unsigned kernel module (which may be incompatible with Secure Boot).
Deploying Ollama with Snap
Snap simplifies the installation of Ollama and its dependencies, ensuring compatibility and streamlined updates. The –beta flag allows you to access the latest features and versions, which might still be under development
sudo snap install --beta ollama
Configuration
Configure Ollama to use the ephemeral disk
sudo mkdir /mnt/models
sudo snap connect ollama:removable-media # to allow the snap to reach /mnt
sudo snap set ollama models=/mnt/models
Installing Mixtral
At this point, you can run one of the open models available out of the box, like mixtral or llama2. If you have a fine-tuned version of these models (a process that involves further training on a specific dataset), you can run those as well.
ollama run mixtral
The first run might take a while to download the model.
Now you can use the model through the console interface:
Installing a UI
This step is optional, but provides a UI via your web browser.
sudo snap install --beta open-webui
Access the web UI securely
To quickly access the UI without open ports in the Azure security group, you can create an SSH tunnel to your VM using the following command:
ssh -L 8080:localhost:8080 ubuntu@${IP_ADDR}
Go to http://localhost:8080 in your web browser on your local machine (the command above tunnels the traffic from your localhost to the instance on Azure).:
In case you want to make this service public, follow this documentation.
Verify GPU usage
sudo watch -n2 nvidia-smi
Check that the ollama process is using the GPU, you should see something like this:
+---------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|===========================================================================|
| 0 N/A N/A 1063 C /snap/ollama/13/bin/ollama 4882MiB |
+---------------------------------------------------------------------------+
Complementary and Alternative Solutions
Charmed Kubeflow: Explore this solution for end-to-end MLOps (Machine Learning Operations), providing a streamlined platform to manage every stage of the machine learning lifecycle. It’s particularly well-suited for complex or large-scale AI deployments.
Azure AI Studio: Provides ease of use for those seeking less customization.
Conclusion
Ubuntu’s open-source foundation and robust ecosystem make it a compelling choice for deploying open-source AI models. When combined with Azure’s GPU capabilities and Confidential Compute features, you gain a flexible, secure, and performant AI solution.
Ubuntu 23.10 experimental image with x86-64-v3 instruction set now available on Azure
Canonical is enabling enterprises to evaluate the performance of their most critical workloads in an experimental Ubuntu image on Azure compiled with x86-64-v3, which is a microarchitecture level that has the potential for performance gains. Developers can use this image to characterise workloads, which can help inform planning for a transition to x86-64-v3 and provide valuable input to the community working to make widespread adoption of x86-64-v3 a reality.
The x86-64-v3 instruction set enables hardware features that have been added by chip vendors since the original instruction set architecture (ISA) commonly known as x86-64-v1, x86-64, or amd64. Canonical Staff Engineer Michael Hudson-Doyle recently wrote about the history of the x86-64/amd64 instruction sets, what these v1 and v3 microarchitecture levels represent, and how Canonical is evaluating their performance. While fully backwards compatible, later versions of these feature groups are not available on all hardware, so when deciding on an ISA image you must choose to maximise the supported hardware or to get access to more recent hardware capabilities. Canonical plans to continue supporting x86-64-v1 as there is a significant amount of legacy hardware deployed in the field. However, we also want to enable users to take advantage of newer x86-64-v3 hardware features that provide the opportunity for performance improvements the industry isn’t yet capitalising on.
Untapped performance and power benefits
Intel and Canonical partner closely to ensure that Ubuntu takes full advantage of the advanced hardware features Intel silicon offers, and the Ubuntu image on Azure is an interim step towards giving the industry access to the capabilities of x86-64-v3 and understanding the benefits that it offers. Intel has made x86-64-v3 available since Intel Haswell was first announced a decade ago. Support in their low power processor family is more recent, arriving in the Gracemont microarchitecture which was first in the 12th generation of Intel Core processors. Similarly, AMD has had examples since 2015, and emulators such as QEMU have supported x86-64-v3 since 2022. Yet, with this broad base of hardware availability, distro support of the features in the x86-64-v3 microarchitecture level is not widespread. In the spirit of enabling Ubuntu everywhere and ensuring that users can benefit from the unique features on different hardware families, Canonical feels strongly about enabling a transition to x86-64-v3 while remaining committed to our many users on hardware that doesn’t support v3. x86-64-v3 is available in a significant amount of hardware, and provides the opportunity for performance improvements which are currently being left on the table. This is why we believe that v3 is the next logical microarchitecture level to offer in Ubuntu, and Michael’s blog post explains in greater detail why v3 should be chosen instead of v2 or v4.
Not just a porting exercise
The challenge with enabling the transition to v3 is that while we expect a broad range of performance improvements depending on the workload, the results are much more nuanced. From Canonical’s early benchmarking we see that certain workloads see significant benefit from the adoption of x86-64-v3; however there are outliers that regress and need further analysis.
Canonical continues to do benchmarking, with plans to evaluate different compilers, compiler parameters, and configurations of hostOS and guestOS. In certain cases, such as the Glibc Log2 benchmark, we have reproducibly seen up to a 60% improvement. On the other hand, we also see other benchmarks that regress significantly. When digging in, we found unexpected behaviour in the compiled code. For example, in one of the benchmarks we verified an excessive number of moves between registers, leading to much worse performance due to the increased latency. In another situation, we noticed a large code size increase, as enabling x86-64-v3 on optimised SSE code caused the compiler to expand it into 17x more instructions, due to a possible bug during the translation to VEX encoding. With community efforts, these outliers could be resolved. However, they will require interdisciplinary collaboration to do so. This also underscores the necessity of benchmarking different types of workloads, so that we can understand their specific performance and bottlenecks. That’s why we believe it’s important to enable workloads to run on Azure, so that a broader community can give feedback and enable further optimisation.
Try Ubuntu 23.10 with x86-64-v3 on Azure today
The community now has access to resources on Azure to easily evaluate the performance of x86-64-v3 for their workloads, so that they can understand the benefits of migrating and can identify where improvements are still required. What is being shared today is experimental and for evaluation and benchmarking purposes only, which means that it won’t receive security updates or other maintenance updates you would expect for an image you could use in production. When x86-64-v3 is introduced for production workloads there will be a benefit to being able to run both v3 and v1 depending on the workload and hardware available. As is usually the case, the answer to the question of whether to run on a v3 image or a v1 image is ‘it depends’. This image provides the tools to answer that cost, power, and performance optimisation problem. In addition to the availability of the cloud image on Azure, we’ve also previously posted on the availability of Ubuntu 23.04 rebuilt to target the x86-64-v3 microarchitecture level, and made available installer images from that archive. These are additional tools that the community can use to benchmark, when cloud environments can’t be targeted.
Earlier this month/week, we announced that you can now benefit from the combined power of Ubuntu Core and Azure IoT Edge to bring the computation, storage, and AI capabilities of the cloud closer to the edge of the network. Azure IoT Edge is a device-focused runtime that enables you to deploy, run, and monitor containerised Linux workloads. Ubuntu Core is a version of Ubuntu that has been specially optimised for IoT and embedded systems. The combination of the two is ideal for those looking for reduced latency, lower bandwidth, and more efficient data processing.
This tutorial helps you get started using an Ubuntu Core device and managing it from the Azure IoT Hub. Azure IoT Hub is the cloud platform which allows you to connect to, configure and deploy Edge workloads directly to your device.
Setting up Ubuntu Core
Ubuntu Core is a minimal, immutable version of Ubuntu focused on providing a reliable and secure platform for connected devices.
Create an Ubuntu SSO Account
Before you can create an Ubuntu Core device, you need to ensure you can connect to it after initial configuration. This will require an Ubuntu SSO account and an SSH keypair.
You can skip this step if you already have an account. If you do not have an SSO account, you can sign up for one here:
In order to authenticate yourself when trying to connect to your Ubuntu Core device, you will need to upload a public SSH key to your SSO account. This will then be automatically downloaded to the Core device during initial configuration.
To generate and upload an SSH key pair, please follow the steps detailed in the link below:
You may also want to come back to this information once you have configured your Core device in the next stage.
Obtaining and configuring an Ubuntu Core Device
For the next stage in the process you will need an IoT device running Ubuntu Core. This can either be a physical device, such as a raspberry pi, or a virtual device on your desktop.
Independently of which option you chose, you should now have a fully working Ubuntu Core device that you can connect to via SSH. You are now ready to provision it for Microsoft Azure.
Installing Azure IoT Edge Snaps
Having created and connected to your Ubuntu Core device, the next step is to install the Azure snaps.
Microsoft provides four snaps for your Ubuntu Core device:
The Identity snap authenticates your device with the Azure cloud.
The Device Agent snap ensures your device is up-to-date.
The Edge snap manages your cloud-deployed workloads on the device.
The Delivery Optimization agent manages downloads of payloads from the Azure cloud.
In addition, Azure’s workloads are distributed as Docker containers and you therefore need to install the Docker Snap to run these.
All five snaps can be installed from your SSH terminal using the following commands:
Note: if you are being asked to use sudo to run snap install, you may need to authenticate yourself with the snap store using sudo snap login <email address>. This will then allow you to perform all snap commands without root privileges.
Wiring up slots and plugs
By default, snaps are dependency-free, untrusted, and strictly confined hence they must be connected to other snaps and system resources once installed using interfaces. Each snap has a selection of plugs and slots that either request or provide certain access. For production deployments, they can be configured to automatically connect to reduce the provisioning workload but to get started you may need to manually configure some of them to ensure they have all the permissions they need.
If installing the snaps from the global snap store, most interfaces will already be connected for you, however there are a few you may need to manually configure.
From an SSH terminal, you can check which interfaces are already connected using the following commands for each snap:
For each snap you will be presented with a list of the interfaces. If the slot is empty, it may need connecting. For example, if you get the following response from snap connections azure-iot-identity:
We can see that some interfaces have been connected but the identity snap can not access hostname information, log information, mount information, system information or the tpm. We need to manually connect them which we can do from our terminal:
The format of this command is snap connect <plug> <slot> but as we are connecting to snapd system slots we do not need to specify them.
IoT Edge
For the IoT Edge agent, we can go through a similar process but this time we also want to connect from one snap (Edge agent) to another (Docker). The following commands should cover all unconnected interfaces.
# Connect to logging and grant permission to query system info
snap connect azure-iot-edge:log-observe
snap connect azure-iot-edge:mount-observe
snap connect azure-iot-edge:system-observe
snap connect azure-iot-edge:hostname-control
# Connect IoT Edge to Docker
snap connect azure-iot-edge:docker docker:docker-daemon
IoT Device agent
undefined
# Connect to logging and grant permission to query system info
snap connect deviceupdate-agent:account-control
snap connect deviceupdate-agent:hardware-observe
#Connect to SnapD
snap connect deviceupdate-agent:snapd-control
#Connect to other Azure Snaps
snap connect deviceupdate-agent:identity-service azure-iot-identity:identity-service
With all the interfaces now connected, we are ready to move onto the connecting to the cloud.
Setting up your Azure IoT Edge account
For the next step, you need to move to the cloud and the Azure IoT Edge portal. If you already have an Azure account, you can sign in here:
You will be given the option to either create a free account (which includes a time-limited, preview credit) or a paid account with access to premium services. Both Azure IoT Hub and Azure IoT Edge are free services that can be used without charge provided you stay within Azure’s usage limitations. More information can be found on this here.
Once you have access to your Azure account and the Azure Portal, you will need to create your IoT Hub. From the Azure services section of the portal, click “More services” and select “IoT Hub” from the “Internet of Things” section.
Once in your IoT Hub, you need to create a Hub. Click the Create button and fill in the details. Once happy, click the ‘Create’ button.
After a brief pause, your Hub will have been deployed and we can now see it in the IoT Hub portal.
Select your Hub and, from the menu on the right hand side of the screen, select Devices.
Click “Add Device”, choose a name for your device and select the “IoT Edge Device” checkbox. Choose any other settings you desire and click “Save”.
Again after a slight pause, your device will have been created and added to your Hub.
Select your device from your Hub and you will be presented with the various options and information. For the moment, we are just interested in the “Primary connection string” as we will need this to provision the actual device.
You can view the connection string by clicking on the small eye icon or copy it to your clipboard by clicking the icon to the right.
Provisioning your Device
In order for your Core device to connect to your newly created IoT Hub, it needs to be
configured with the connection string we have just obtained.
Returning to the SSH terminal of your Core device, create a file called config.toml.
At this point it may be useful to install a text editor onto your Core device. Follow the steps below to install the nano strictly confined snap and connect it to your home directory. Then open the config.toml file to edit
Apply that configuration file to your Azure Snap using the following command:
sudo snap set azure-iot-edge raw-config="$(cat config.toml)"
It is also possible to authenticate your device to the Azure IoT Hub using X.509 certificates. For information on how to use that method, please refer to this documentation from Microsoft.
Your Device in Azure IoT Hub
Once configured, your device will download some containers to allow it to run Azure IoT Edge workloads. This may take some time depending on your network connection speed but once complete your device will be visible from your Azure portal and you can configure it with additional workloads from there as well as explore all the offerings Azure has for your device.
You should now have a fully working and configured Ubuntu Core device which can be remotely managed with the Azure IoT Hub. From here you can explore the features Azure IoT has to offer.
If you want to try and deploy your first module to your Edge device, this tutorial from Microsoft shows you how you can deploy a sensing module that will send simulated telemetry data from your device to the cloud. It is the perfect place to get started with your Ubuntu Core Azure IoT Edge device.
For more information on what you can do with Azure IoT, please refer to the Microsoft documentation.
London, 20 March 2024. Canonical has announced that Ubuntu Core, its operating system optimised for the Internet of Things (IoT) and edge, has received Microsoft Azure IoT Edge Tier 1 supported platform status from Microsoft. This collaboration brings computation, storage, and artificial intelligence (AI) capabilities in the cloud closer to the edge of the network.
The power of the cloud on the edge
Azure IoT Edge enables businesses to remotely and securely deploy and manage cloud-native workloads directly on their IoT devices, at scale, and with robust observability.
With the ability to deploy and manage containerised applications on devices, teams can process data, run machine learning models, perform analytics, and carry out other tasks right at the edge of the network. This approach helps reduce latency, conserve bandwidth, and it provides more immediate insights from data near to where it is generated. It is especially useful in scenarios where real-time decision-making is crucial, where network connectivity might be unreliable, or where data privacy and security concerns demand local data processing.
The security of Ubuntu Core
Ubuntu Core is an operating system designed specifically for the IoT and embedded devices. Its range of features make it ideal for secure, reliable, and scalable deployments. Built on the power of Snaps, Ubuntu Core provides a minimal core with support for multiple architectures and types of devices. Security is baked-in with secure boot and full disk encryption, and over-the-air (OTA) transactional updates to ensure that devices are always up to date. Coupled with Canonical’s Long Term Support, which offers up to 10 years of maintenance and security updates, Ubuntu Core provides long-term peace of mind for IoT implementations.
With the introduction of the Azure IoT Edge Snaps suite, the process of deploying edge workloads to the extensive array of devices and architectures supported by Ubuntu Core has become a streamlined, seamless, experience. Combined with the ability to remotely manage and configure both the processing and system components of fleets of devices directly from Azure, teams benefit from robust security and optimised performance.
“With Microsoft committing their support for Ubuntu Core with the release of the Microsoft Azure IoT Edge Snaps we see another example of the industry’s enthusiasm to adopt the operating system to fulfil all of their IoT needs. We look forward to growing this relationship further with Microsoft in the future”. – Michael Croft-White, Engineering Director.
“In collaboration with Canonical, we are making it simpler to reliably connect devices to Microsoft Azure IoT services. Snap support in Azure IoT Edge helps ensure consistent performance, enhanced security, and efficient updates across Linux distributions that support Snaps.”
Kam VedBrat, GM, Azure IoT
Further reading
More information on Ubuntu Core can be found at ubuntu.com/core. Our “Intro to Ubuntu Core 22” webinar is a comprehensive resource for everything you need to know about Ubuntu Core.
If you are not already familiar with Microsoft’s Azure IoT Edge, more information can be found here.
Are you interested in running Ubuntu Core with Azure IoT on your devices and are working on a commercial project?
Canonical, the publisher of Ubuntu, provides open-source security, support and services. Our portfolio covers critical systems, from the smallest devices to the largest clouds, from the kernel to containers, from databases to AI. With customers that include top tech brands, emerging startups, governments and home users, Canonical delivers trusted open source for everyone.
À la mi-janvier 2024, Google avait pris la décision de supprimer les frais de sortie pour les clients de ses services Cloud offerts via Google Cloud Platform. Désormais, c'est au tour de Microsoft d'adopter une politique similaire pour son Cloud Azure. Faisons le point sur cette annonce !
Sur son site dédié aux mises à jour de services Azure, Microsoft a "discrètement" mis en ligne un court message, qui pourtant contient une information cruciale : la fin des frais de sortie pour les utilisateurs qui souhaitent migrer leurs données présentes sur Azure vers un autre fournisseur Cloud. Une décision qui va dans le sens de la décision prise par Google en janvier 2024, suivi par Amazon Web Services en février 2024 : il ne manquait plus que Microsoft pour compléter ce trio américain.
À l'instar de Google, Microsoft est clair : si vous partez, et que vous souhaitez partir gratuitement, vous devez migrer l'intégralité de vos services et arrêtez d'utiliser les services Azure, en clôturant votre compte. Autrement dit, si vous sortez du Cloud Azure uniquement une partie de vos services, vous ne pourrez pas bénéficier de cette gratuité.
L'entreprise américaine précise : "Le support Azure appliquera le crédit lorsque le processus de transfert des données sera terminé et que tous les abonnements Azure associés au compte auront été annulés." - À l'inverse, Amazon avec son Cloud AWS n'impose pas la fermeture du compte.
Microsoft en profite pour rappeler qu'il y a déjà une gratuité, partielle, pour quitter Azure : "Azure offre déjà les premiers 100 Go/mois de données transférées gratuitement à tous les clients dans toutes les régions Azure du monde."
Si Microsoft, Google et Amazon ont pris une telle décision, ce n'est probablement pas par plaisir ni par générosité, mais certainement pour se mettre en conformité avec un nouveau texte européen qui doit entrer en vigueur en septembre 2025 : le Data Act. Ceci est d'autant plus vrai que le Digital Markets Act est entré en vigueur récemment, et qu'il contient aussi des directives qui vont en ce sens. Ceci est une première étape pour offrir plus de liberté et de souplesse aux clients d'Azure.
With Ubuntu confidential AI on Azure, businesses can undertake various tasks including ML training, inference, confidential multi-party data analytics, and federated learning with confidence.
The effectiveness of AI models depends heavily on having access to large amounts of good quality data. While using publicly available datasets has its place, for tasks like medical diagnosis or financial risk assessment, we need access to private data during both training and inference.
When performing machine learning tasks in the cloud, enterprises understandably have concerns about the potential compromise of their sensitive data privacy as well as their model’s intellectual property. Additionally, stringent industry regulations often prohibit the sharing of such data. This makes it difficult, or outright impossible, to utilise large amounts of valuable private data, limiting the true potential of AI across crucial domains.
Confidential AI tackles this problem head on, providing a hardware-rooted execution environment that spans both the CPU and GPU. This environment enhances the protection of AI data and code at runtime by helping to safeguard it against privileged system software (such as the hypervisor or host OS) and privileged operators in the cloud.
Confidential AI is made possible thanks to confidential computing, a game-changing technology that represents a significant departure from the traditional threat model of public clouds. In the past, vulnerabilities within the extensive codebase of the cloud’s privileged system software, including the operating system, hypervisor, and firmware, posed a constant risk to the confidentiality and integrity of running code and data. Similarly, unauthorised access by a malicious cloud administrator could compromise the security of your virtual machine (VM) and its platform.
Ubuntu CVMs are here to give you back control over the security guarantees of your VMs. They enable you to run your workload within a hardware-protected Trusted Execution Environment, TEE. Such secure and isolated environments are purpose-built to prevent unauthorised access or alterations to applications and data at run-time, thereby enhancing security for organisations managing sensitive and regulated data.
As such, CVMs’ primary goal is to safeguard your guest workloads from various potential software threats, including the virtual-machine manager and other non-CVM software on the platform. CVMs also enhance your workload’s security against specific physical access attacks on platform memory, including offline dynamic random access memory (DRAM) analysis such as cold-boot attacks and active attacks on DRAM interfaces.
From confidential computing to confidential AI
While confidential computing efforts have historically focused primarily on CPUs, the advent of Nvidia H100 GPUs with confidentiality computing capabilities opens up new possibilities for extending this security paradigm to GPUs as well. The Azure solution, which integrates both CPU and GPU components, is what makes confidential AI achievable. At a high level, this solution relies on the following components:
CPU-TEE: Ubuntu confidential VMs that run on the AMD 4th Gen EPYC processors with SEV-SNP protect the workload’s computation while in the CPU:
Run-time confidentiality: the DRAM of your Ubuntu CVMs is kept encrypted thanks to the new AES-128 hardware encryption engine that sits within the CPU memory memory controller. This engine encrypts and decrypts memory pages whenever there is a memory read or write operation. Instead of having workload code and data in plain text in system memory, they are encrypted using a hardware-managed encryption key. This encryption and decryption process happens seamlessly within the CPU, ensuring strong memory isolation for confidential workloads.
Run-time Integrity: Ubuntu CVMs make use of the new AMD SEV SNP instructions and data structures that allow auditing of security-sensitive tasks typically carried out by privileged system software. These tasks encompass memory management and access to platform devices. For example, when reading memory pages mapped to confidential workloads, these new instructions also provide information about the last value written into the page. This feature helps prevent data corruption and replay attacks by detecting unauthorised modifications to memory pages.
GPU-TEE: NVIDIA H100 Tensor Core GPUs, which protect the confidentiality and integrity of the workload’s computation within the GPU.
Encrypted PCIe communication between the CPUs and GPU.
Attestation: Enables a relying party, whether it’s the owner of the workload or a user of the services provided by the workload, to cryptographically verify the security claims of both the CPU and GPU TEEs.
By integrating these components into a cohesive solution, confidential AI becomes not only feasible but also practical, allowing organisations to harness the power of AI while maintaining the highest standards of data security and confidentiality. Confidential AI can then be further augmented with cryptographic primitives, such as differential privacy, which protect the workload from further sophisticated data leakage.
Build your confidential AI workloads with Ubuntu today
Confidential AI can support numerous use cases across the entire lifecycle of building and deploying an AI application. For example, you can use Ubuntu CVMs during the training phase to protect your data, model IP, and its weights.
Confidential AI can also be beneficial for fine-tuning large language models, whereby enterprises need to use private data to optimize the generic models and improve their performance for their specific industries
We firmly believe that confidential AI represents a pivotal opportunity to unleash the full potential of AI, especially for industries that need to deal with security-sensitive data, such as healthcare and finance. We invite you to join us on this transformative journey with Ubuntu. Together, we can chart new horizons in AI innovation while steadfastly maintaining the highest standards of privacy and security for sensitive data.
Connexion
