GPU Isolation Mechanisms for Multi-Tenant AI Workloads

GPU isolation for multi-tenant AI workloads is not one setting, it is a stack made up of hardware partitioning, a virtualization layer, and an orchestration layer sitting on top, each with its own guarantees and its own failure modes. Get the layers confused, and a team ends up with a MIG slice that isolates memory beautifully but a container runtime beneath it that hands out root to anyone who knows the right vulnerability to exploit. This piece works through each layer in order, what it actually protects against, where it breaks, and what it costs.

The stakes are not abstract. Ninety percent of organizations now deploy AI systems in some form, yet only five percent say they feel confident in their security readiness for those systems, and ninety-seven percent of breached organizations lacked proper AI access controls. A GPU running inference on proprietary model weights is not a commodity compute resource being time-shared. It's temporarily holding the most valuable asset the organization has, and the isolation model around it needs to reflect that.

Start with a distinction that trips up a lot of platform teams: GPU isolation is not CPU isolation with a different device driver. A CPU can be sandboxed inside a namespace, cgroup, or a lightweight VM with fairly mature tooling from a mainstream operating system. A GPU is a PCIe device that does its own memory management, its own scheduling, and talks to the host over a DMA path that a container namespace has no visibility into. Isolating it properly means invoking IOMMU technology (Intel VT-d or AMD-Vi) to enforce DMA isolation, binding the device through VFIO, keeping IOMMU groups separated, and running a VMM that actually supports PCIe passthrough. None of that comes for free just because a workload is "in a container." A GPU-enabled container shares the host kernel and the host's NVIDIA driver namespace with every other tenant on that box. There is no boundary there by default, full stop.

The three sharing models and what each one isolates

Three mechanisms let multiple tenants share a single physical GPU, and they sit on a real hierarchy of guarantees, not just a menu of equally valid options.

Time-slicing is the simplest and the weakest. It's a purely software-based scheme: workloads take turns, each getting the full GPU for a short window before yielding to the next. There's no hardware partitioning happening anywhere. Every job shares the same GPU memory and the same compute resources, and nothing stops one workload's memory footprint or a runaway kernel from affecting the next tenant's window. It works on a range of CUDA-capable NVIDIA GPUs, and it's enabled through a Kubernetes ConfigMap. That simplicity is why it belongs in development environments or internal tooling where the tenants sharing the card already trust each other. It is not a security boundary. Calling it one is the mistake.

MPS, NVIDIA's Multi-Process Service, is a step up in a meaningful way: it lets multiple CUDA processes execute kernels concurrently, actual simultaneous execution rather than time-slicing's rapid turn-taking. But the isolation story depends heavily on which GPU generation is running it. On pre-Volta hardware, MPS clients shared a single GPU address space, which meant a memory fault in one process could cross into another process's territory. That changed with Volta and every architecture since (Ampere's A100, Hopper's H100 and H200): each MPS client now gets its own fully isolated address space. So MPS on modern hardware gives real concurrent execution with genuine address-space separation, even though it still falls short of the physical separation MIG provides.

MIG, Multi-Instance GPU, is the hardware answer. It partitions a single physical GPU into instances that get their own dedicated streaming multiprocessors, their own L2 cache slice, their own memory controllers, and their own HBM address bus. It's actual physical separation enforced at the hardware level, not a clever scheduling trick layered on shared silicon. It's actual physical separation enforced at the hardware level.

Line those three up and the hierarchy is plain: time-slicing offers access without isolation, MPS offers concurrent execution with address-space separation on one generation of hardware and later, and MIG offers hardware-enforced separation with guaranteed quality of service. Picking among them is really picking how much you trust the tenants sharing the card, and how much that trust is worth risking.

How MIG partitioning works at the hardware level

MIG debuted with the Ampere architecture and now runs across a wide swath of NVIDIA's data center and workstation lineup: A100, A30, H100, H200, B200, GB200, and the RTX PRO 6000 Blackwell card. The A100, H100, H200, B200, and GB200 all support up to seven MIG instances per GPU; the smaller A30 tops out at four.

What actually makes this hardware isolation rather than software isolation is the memory path. Each MIG partition gets its own separate route through the entire memory subsystem, meaning unique assignment of on-chip crossbar ports, L2 cache banks, memory controllers, and DRAM address buses to each instance. One tenant physically cannot read or overwrite another tenant's GPU memory, because the silicon routes their traffic through entirely different components. That is a materially different guarantee than "the scheduler promises not to let that happen."

From the operating system's point of view, each MIG instance shows up as its own distinct GPU device. That means each one can run its own container completely independently, with its own device node and its own resource limits, which is exactly the abstraction Kubernetes device plugins are built to consume.

Partition an H100 SXM5 into the full seven-way MIG configuration and each instance gets roughly 10 GB of HBM3 and about 2,048 CUDA cores. That's a real, usable GPU slice, not a sliver. Seven independent tenants, each with hardware-guaranteed memory isolation, off one physical card. For a lot of inference workloads, that's more than enough headroom.

Where MIG breaks down: fixed profiles, fragmentation, and reconfiguration cost

MIG's hardware guarantees come at the cost of flexibility, and the Flex-MIG research lays out four specific constraints that show exactly where that cost appears.

Fixed profiles are the first wall. MIG only supports a predefined set of instance shapes, so a workload needing something between the standard sizes just doesn't get it. There's no arbitrary sizing knob to turn.

Tree-constrained merging is the second. MIG's resource layout is structured as a tree, and that structure means smaller instances often cannot be recombined into a larger one even when the physical capacity is sitting right there, unused, on the same card. Fragmentation becomes a real operational problem: seven small instances that individually can't be merged back into one large instance, even though nothing else is running on the GPU.

The third constraint is that MIG instances are scoped to a single physical GPU, which imposes a hard ceiling on how large any individual job can grow within one partition.

The fourth is more operational than architectural, but it's the one that actually bites in production: reconfiguration cost. Changing a MIG layout means stopping every process on that GPU, tearing down the existing instances, building new ones, and restarting all affected workloads. In a Kubernetes cluster, that's a node drain. And the timescale mismatch here is severe: reconfiguration operates on a timescale of minutes, while inference requests operate on a timescale of milliseconds. That gap is a critical mismatch for any production inference pipeline that needs to flex its GPU allocation in response to shifting demand.

There's no dynamic scaling path around this either. A workload that outgrows its MIG slice has to be migrated to a bigger slice, and that migration runs through the same full reconfiguration cycle described above. Add to that a constraint specific to the major clouds: AWS, GCP, and Azure all lock MIG configuration at the hypervisor layer, so tenants renting GPU instances on these platforms cannot change MIG profiles at runtime even if their workload's needs shift mid-session. Tenants renting GPU instances on these platforms cannot change MIG profiles at runtime even if their workload's needs shift mid-session, and that hardware isolation is enforced by locking MIG configuration at the hypervisor layer. The operational rigidity around that locked configuration matches the strength of the hardware isolation itself.

vGPU and SR-IOV: the virtualization-layer alternatives

Where MIG partitions silicon, NVIDIA's vGPU software takes a different approach and partitions at the virtualization layer instead. The hypervisor virtualizes the physical GPU and hands out slices to multiple VMs, each of which perceives its own virtualized portion of the card, and each of which sits behind full IOMMU protection.

What vGPU adds that MIG's static hardware partitions cannot is operational flexibility: the ability to run mixed workloads, virtual desktop infrastructure and compute jobs, on the same infrastructure at the same time. That is a genuine advantage in environments where the workload mix shifts over time and a fixed hardware partition would need constant reconfiguration to keep up.

The utilization numbers back this up in a fairly dramatic way. VMware's deployment of vGPU across ten thousand hosts hit 82 percent utilization, against 34 percent for dedicated, unshared GPU hardware doing comparable work. That's not a marginal efficiency gain; a fleet that's mostly idle becomes one that's mostly working.

SR-IOV takes yet another angle: hardware-assisted I/O virtualization, where a physical function manages the GPU's resources and configuration while virtual functions give VMs direct hardware access. Because the virtual functions talk to hardware queues directly, there's no software layer intercepting every command submission, which strips out the overhead that a fully virtualized device usually carries. Memory isolation between tenants comes from DMA remapping enforced by the same class of IOMMU hardware that underpins MIG and vGPU. Intel's SR-IOV deployment reached 96 percent of bare-metal performance on compute workloads, which is about as close as virtualized hardware gets to running unvirtualized.

The GPU sandbox layer: why hardware partitioning alone is not enough

A distinction that sounds pedantic actually matters a great deal in practice. A MIG instance is a hardware partition. A GPU sandbox is something else: an isolated execution environment that layers hardware virtualization or syscall interception on top of whatever partitioning already exists, specifically to stop a workload from reaching out and touching the host system or another tenant's process.

Why would hardware partitioning alone not be enough? Because MIG's guarantee is about memory and compute paths inside the GPU. It says nothing about what a process can do to the host kernel, the filesystem, or the network stack once it's running. A MIG slice with a compromised container runtime sitting on top of it is still a MIG slice with a compromised container runtime sitting on top of it. The hardware isolation and the execution isolation are answering different questions.

This matters most for a specific set of platforms: multi-tenant services where tenants submit GPU-accelerated code directly, AI agents invoking local inference or embedding generation on behalf of untrusted input, platforms running user-submitted training jobs or RL reward evaluation at real scale, and code execution products that let users attach a GPU to a notebook session. In every one of these, the workload itself is not trusted, which is a fundamentally different threat model than "which internal team gets which MIG slice."

The available sandboxing path splits cleanly on one question: is nested virtualization available on the host? When it is, microVM-based isolation through KVM or Kata Containers becomes possible, with the GPU passed through into a sandboxed runtime that has its own kernel. That's hardware-level isolation, and it's the strongest option on the table. When nested virtualization is not available, which describes most cloud VM environments, the fallback is gVisor, where the isolation boundary moves from hardware virtualization down to the syscall level. It's a different, weaker guarantee, but it's what's available when the underlying infrastructure won't support a nested hypervisor.

Firecracker excludes GPU passthrough as a deliberate design decision, implementing only a minimal set of emulated devices to keep its attack surface as small as possible. That's a defensible security posture for what Firecracker is built for, but it also means platforms built on Firecracker are CPU-only by design. Any product needing GPU-accelerated multi-tenant sandboxing has to look elsewhere.

Known escape vectors: CVEs and prompt-injection chains that breach GPU isolation

Hardware partitioning and sandboxing describe what the architecture is supposed to guarantee. What actually happens when that architecture has a bug is a separate, and instructive, story.

NVIDIA disclosed seven security vulnerabilities in its GPU Display Driver bulletin on January 27, 2025. One of them, CVE-2025-23266, was disclosed separately in July 2025 and turned out to be serious: it allowed a malicious actor to bypass Container Toolkit isolation entirely and gain root access to the host system. It's a software-stack vulnerability, sitting above the MIG hardware layer, that undermines isolation even when the MIG configuration underneath it is completely correct. A perfectly partitioned GPU does not help if the toolkit managing container access to that partition has a hole in it.

CVE-2024-21626, nicknamed "Leaky Vessels," is a different kind of failure: a file descriptor leak in the runc container runtime that let an attacker escape the container and reach the host filesystem. That one is unrelated to GPU configuration; it's a container-layer bug, but it runs in the same category of consequence: a workload assumed to be isolated turns out not to be, because the isolation depended on something other than the GPU.

Then there's the newer, less familiar category: AI-specific escape chains that ride in through prompt injection rather than a driver bug. An incident involving Snowflake's Cortex Code CLI in early 2026 combined indirect prompt injection with weak command validation, and the result let AI-generated instructions bypass a human-in-the-loop approval step and escape sandbox mode, ultimately enabling arbitrary code execution and access to cached credentials. Microsoft's security report documented a related pattern: remote code execution vulnerabilities in the Semantic Kernel AI agent framework, reached through a chain that started with prompt injection and ended in shell execution.

These two incidents point at something the earlier CVEs do not: the prompt injection surface is now an isolation escape vector in its own right. Traditional GPU isolation, MIG's hardware partitions, vGPU's hypervisor boundary, gVisor's syscall interception, was designed against a threat model where the attacker is a malicious workload or a malicious tenant. None of those mechanisms were built with the assumption that the attacker might be a carefully worded string embedded in a document that an AI agent reads and then acts on. That's a genuinely different attack surface, and it sits on top of, not instead of, the hardware and virtualization boundaries already discussed.

Orchestration: how Kubernetes surfaces, schedules, and governs GPU isolation at cluster scale

None of the isolation mechanisms above matter much at scale unless something schedules and governs them across a cluster, and for the vast majority of multi-tenant GPU deployments today, that something is Kubernetes.

MIG instances, since they appear to the OS as distinct GPU devices, get exposed to Kubernetes through device plugins, and the standard pattern for multi-tenancy is allocation by namespace, one MIG slice mapped to one tenant namespace. It's a clean model precisely because MIG already did the hard work of hardware separation, leaving Kubernetes to handle bookkeeping rather than enforcement.

Time-slicing looks superficially similar in Kubernetes but behaves very differently underneath. An administrator defines a set of replicas for a given GPU, and each replica gets handed out independently to a pod, the same way any other resource request works. But unlike MIG, none of those replicas carry memory or fault isolation from each other. If one pod crashes, or misbehaves in a way that corrupts GPU state, that can affect every other pod sharing the same physical card. Kubernetes will schedule work onto time-sliced replicas just as readily as it schedules onto MIG slices, but time-sliced replicas share memory and fault state while MIG slices do not, and that distinction is easy to lose in a device-plugin configuration file that treats them as interchangeable resource types.

The reconfiguration cost from MIG resurfaces here in concrete operational terms. Changing MIG profiles on a running Kubernetes node means draining that node first, evicting every pod on it before the GPU can be repartitioned. At cluster scale, with dozens or hundreds of GPU nodes, that's a real scheduling and availability cost, not just an inconvenience. It's part of why dynamic partitioning platforms, Run:ai among them, have emerged specifically to manage this friction, sitting between the raw MIG API and the cluster scheduler to make reconfiguration less disruptive.

Resource quotas and Kubernetes' service tiers for guaranteed, burstable, and best-effort workloads map onto GPU allocation strategy the same way they map onto CPU and memory allocation, giving operators a lever to decide which tenants get guaranteed access to their requested GPU slice versus which tenants get best-effort access that can be reclaimed under pressure. LinkedIn's use of QoS enforcement in its own infrastructure held a 99.9 percent SLA compliance rate, a data point that suggests the orchestration layer, done carefully, can make good on isolation promises the hardware and virtualization layers set up beneath it.

Taken together, the picture across all seven layers is less about which single mechanism is "best" and more about matching guarantees to threat model: time-slicing for trusted internal teams, MIG for hard multi-tenant memory isolation, vGPU or SR-IOV where flexibility and utilization matter as much as isolation, a sandbox layer wherever the workload itself cannot be trusted, and an orchestration layer that actually treats a hardware partition and a software timeslice as separate guarantees. Skipping a layer, or assuming one layer's guarantee substitutes for another's, is exactly the gap the CVEs and the prompt-injection chains above have already shown a willingness to exploit.