POST-TRAINING & DISTILLATION

MODULE 8: ADVANCED CAPABILITIES & DEPLOYMENT · SCENE 24

Deployment: Distillation & Serving Prep

Teacher-student KL distillation, FP8 calibration curves, and the pipeline from checkpoint to serving.

DISTILL KL · 0.0158T² · SOFTENING 65%FP8 CALIB KL · 0.042 (20 SAMPLES)
NEXT-TOKEN DISTRIBUTION · STUDENT CHASING THE TEACHERtok 1tok 2tok 3tok 4tok 5■ TEACHER■ STUDENT (CONVERGING)DEPLOYMENT PIPELINEDISTILL→QUANTIZE 4-BIT→CALIBRATE FP8→EVAL EXPORT

RUN: the student chases the teacher's softened distribution, then the artifact walks the deployment pipeline.

ARCHITECTURE & FORMULATIONModule 8: Advanced Capabilities & Deployment

Deployment: distillation and serving prep

The last mile converts a good model into a shippable one: distill capability into a smaller student, calibrate for FP8 serving, and verify the deployed artifact reproduces the evaluated behavior — not a nearby cousin.

KL Distillation

The student minimizes KL against the teacher's softened distribution (temperature T), inheriting the teacher's dark knowledge — relative probabilities — not just its argmax.

Calibration Sets

FP8 serving quantization uses a few hundred representative samples to fix activation ranges. Too few samples inflates output KL versus the reference.

Evals on the Shipped Artifact

Re-run the full eval suite on the quantized, exported model. The artifact you serve is the artifact you score.

MATHEMATICAL FORMULATION · DISTILLATION LOSS & SERVING MEMORY
L = T² · KL( softmax(z_t/T) ‖ softmax(z_s/T) ); VRAM_GB = params_B · bits/8 · (1 + overhead)

The T² factor keeps gradient magnitudes comparable across temperatures. A 7B model at 4-bit needs ~3.5 GB of weights — plus 15–40% for KV cache and runtime overhead depending on batch and context.

REAL-WORLD PRODUCTION ENGINEERING
  • Sequence-level distillation (train the student on the teacher's sampled outputs) usually beats pure logit distillation for aligned behavior.
  • Ship with the calibration recipe recorded: samples, quantization scheme, and the eval delta vs bf16 — reproducibility is part of the artifact.
← 23 · Post-Training Quantization: GPTQ, AWQ, FP8
ATLAS →
Agent-native web · research mode

Experimental WebMCP · evolving community-group draft, not a W3C Standard or on the Standards Track.

Native status: Initializing. Registered: none

DOM → inferred capabilities · WebMCP → declared capabilities. Tools read real CUDA/Inference Atlas content. Calls may highlight temporary UI state. No backend or remote telemetry.

Local harness verifies application logic; it does not demonstrate native discovery or a browser agent.

Input schema
{
  "type": "object",
  "properties": {
    "query": {
      "type": "string",
      "minLength": 1,
      "maxLength": 300
    },
    "limit": {
      "type": "integer",
      "minimum": 1,
      "maximum": 10
    }
  },
  "required": [
    "query"
  ],
  "additionalProperties": false
}
Local log: last 100 calls while this panel is open. Inputs may contain personal text; review before sharing.
 
[]