INFRASTRUCTURE

MODULE 4: CLUSTER SCHEDULING & ORCHESTRATION · SCENE 16

MIG hardware slicing vs time-sliced virtualization

Partitioning an H100 into up to 7 isolated GPU instances with dedicated SMs, memory controllers, and L2 cache crossbars for guaranteed QoS.

STREAMING MULTIPROCESSORS
14 SMs (of 132)
ISOLATED HBM CAPACITY
10 GB
MEMORY BANDWIDTH
420 GB/s

Up to 7 concurrent isolated instances can run on a single physical 80GB H100 with zero noisy-neighbor interference.

ARCHITECTURE & FORMULATIONModule 4: Cluster Scheduling & Orchestration

MIG Hardware Slicing vs Time-Slicing

Multi-Instance GPU (MIG) technology enables a single physical GPU (such as an A100 or H100) to be partitioned into up to 7 completely isolated hardware instances. Each instance has dedicated Streaming Multiprocessors (SMs), memory controllers, and L2 cache crossbars for deterministic Quality of Service (QoS).

Hard Hardware Partitioning vs Time-Slicing

Software time-slicing shares SMs and cache across processes, causing cache thrashing and unpredictable p99 latency spikes when co-located jobs compete. MIG provides physical hardware boundaries with zero cross-tenant interference.

MIG Instance Profiles (1g.10gb to 7g.80gb)

Profiles define the exact allocation of hardware resources. For example, 1g.10gb allocates 14 SMs, 10 GB HBM, and 1/7th of total memory bandwidth—ideal for lightweight inference or embedding models.

Multi-Tenant Security & Fault Isolation

A fatal error or out-of-memory (OOM) crash in one MIG slice terminates only that specific instance without affecting other active slices running on the same physical silicon.

MATHEMATICAL FORMULATION · MIG RESOURCE SLICING INVARIANT
∑ Profile_SM,k ≤ SM_total and ∑ Profile_HBM,k ≤ HBM_total

On an H100 (132 enabled SMs, 80GB HBM), instances are allocated across 7 hardware partitions (e.g., two 3g.40gb slices or seven 1g.10gb slices), guaranteeing deterministic memory bandwidth to each tenant.

REAL-WORLD PRODUCTION ENGINEERING
  • Use NVIDIA GPU Operator for Kubernetes to dynamically provision MIG devices as native k8s extended resources (nvidia.com/mig-1g.10gb).
  • MIG does not support NVLink peer-to-peer transfers between slices on the same GPU; use full GPUs for distributed training and MIG for inference serving.
  • Monitor MIG instance utilization via nvidia-smi mig -lgi to prevent under-utilized static partitions.
← 15 · Straggler detection & collective barrier lag
17 · Silent Data Corruption (SDC) & transient bit-flips →
Agent-native web · research mode

Experimental WebMCP · evolving community-group draft, not a W3C Standard or on the Standards Track.

Native status: Initializing. Registered: none

DOM → inferred capabilities · WebMCP → declared capabilities. Tools read real CUDA/Inference Atlas content. Calls may highlight temporary UI state. No backend or remote telemetry.

Local harness verifies application logic; it does not demonstrate native discovery or a browser agent.

Input schema
{
  "type": "object",
  "properties": {
    "query": {
      "type": "string",
      "minLength": 1,
      "maxLength": 300
    },
    "limit": {
      "type": "integer",
      "minimum": 1,
      "maximum": 10
    }
  },
  "required": [
    "query"
  ],
  "additionalProperties": false
}
Local log: last 100 calls while this panel is open. Inputs may contain personal text; review before sharing.
 
[]