Unit economics bridges the gap between infrastructure engineering and product viability. Calculating the exact cost per million tokens for both pretraining runs and inference serving endpoints enables disciplined capacity planning and margin modeling.
Pretraining Token Cost Formula
Pretraining cost is directly proportional to model parameter count, training token volume, and GPU rental rate, inversely proportional to hardware MFU.
Inference Serving Cost Structure
Inference unit economics depends on time-to-first-token (TTFT prefill cost), inter-token latency (decode memory bandwidth cost), batch size efficiency, and KV cache memory capacity.
FinOps Margin Optimization
Improving training MFU from 35% to 52% on a 15-trillion token run saves over $2.5M in compute bills; optimizing inference batching cuts serving cost per user request by 4x.
MATHEMATICAL FORMULATION · PRETRAINING TOTAL COST FORMULATION
Training an 8B model on 15T tokens on H100 GPUs ($2.00/hr, 989 TFLOPs peak) at 50% MFU: Total GPU-hours = (6 * 8e9 * 15e12) / (989e12 * 0.50 * 3600) = 404,448 hours * $2.00 = $808,897 ($0.054 per 1M training tokens).
REAL-WORLD PRODUCTION ENGINEERING
Continuously log cost-per-token metrics to business intelligence dashboards to detect cost regressions in fine-tuning pipelines.
Use vLLM or TensorRT-LLM continuous batching and PagedAttention to maximize inference token throughput per dollar.
Implement prompt caching for system prompts and repetitive contexts to eliminate redundant prefill FLOPs.