POST-TRAINING & DISTILLATION

MODULE 1: SFT FOUNDATIONS · SCENE 03

LoRA & QLoRA: Math and Memory Layouts

Rank-r adapters, trainable-parameter share, and the 4-bit base + bf16 LoRA memory bill vs. a full fine-tune.

RANK rr=16ADAPTERS · 54.7M PARAMS
VRAM (GB) · 7B MODELFULL FINE-TUNE · 112 GBQLoRA (4-BIT BASE + ADAPTERS) · 4.7 GB24× SMALLER · FITS ON ONE 24GB CONSUMER GPU ✓

Frozen NF4 base (~3.9 GB) plus bf16 rank-16 adapters. Merged at deploy, inference is identical to the original model.

ARCHITECTURE & FORMULATIONModule 1: SFT Foundations

LoRA & QLoRA: the parameter-efficient memory trade

Instead of updating every weight, LoRA freezes the base model and learns tiny low-rank matrices whose product approximates the needed weight change. QLoRA stacks 4-bit quantization of the frozen base on top, turning a multi-GPU job into a single-GPU job.

Low-Rank Adapters

The weight update ΔW is factorized as B·A where A is r×d and B is d×r. With r ≪ d, trainable parameters collapse to a fraction of a percent.

4-bit Base

QLoRA stores the frozen base in NF4 (~0.5 bytes/param) with double quantization and paged optimizers, while LoRA adapters stay in bf16.

Merge at Deploy

After training, W' = W + B·A merges back into the base weights, so inference cost is identical to the original model — zero latency overhead.

MATHEMATICAL FORMULATION · LORA MEMORY LEDGER
QLoRA_GB ≈ params_B × 0.55 + lora_params_M × 16B/param ÷ 10⁹

A 7B model needs ~3.9 GB for the 4-bit base plus a few GB of adapters and optimizer state; a full fine-tune of the same model needs ~7 × 16 = 112 GB.

REAL-WORLD PRODUCTION ENGINEERING
  • r=8–64 on q_proj/k_proj/v_proj/o_proj + MLP is the practical sweet spot; higher rank helps style-heavy domains more than factual recall.
  • QLoRA made it routine to fine-tune a 33B model on a single 24GB consumer GPU — the democratization moment for instruction tuning.
← 02 · SFT Loss: Prompt Masking & Completion Targets
04 · Data Curation & Quality Filtering →
Agent-native web · research mode

Experimental WebMCP · evolving community-group draft, not a W3C Standard or on the Standards Track.

Native status: Initializing. Registered: none

DOM → inferred capabilities · WebMCP → declared capabilities. Tools read real CUDA/Inference Atlas content. Calls may highlight temporary UI state. No backend or remote telemetry.

Local harness verifies application logic; it does not demonstrate native discovery or a browser agent.

Input schema
{
  "type": "object",
  "properties": {
    "query": {
      "type": "string",
      "minLength": 1,
      "maxLength": 300
    },
    "limit": {
      "type": "integer",
      "minimum": 1,
      "maximum": 10
    }
  },
  "required": [
    "query"
  ],
  "additionalProperties": false
}
Local log: last 100 calls while this panel is open. Inputs may contain personal text; review before sharing.
 
[]