Skip to content

Agent Infrastructure: trajectory runner (Exp12)

Paper Section 5.2.3 (rollout WAL), Section 5.2.5 (DSec trajectory logs) · Code: src/deepseek_v4_lab/agents/trajectory.py

Evidence tags: PAPER stated by DeepSeek · DERIVED mathematically implied · IMPLEMENTED reproduced in this project · MEASURED observed in our experiments · INFERRED our interpretation · UNKNOWN not established. Full definitions: evidence ledger

Why this exists

Problem: when agent trajectories become training data, killing a rollout mid-flight either wastes it or biases the distribution (short trajectories survive more often).

Response: write-ahead trajectory log + checkpointed sandbox state; resume fast-forwards positionally and never re-executes logged commands.

New cost: runtime must be deterministic enough for positional replay; log integrity becomes a training-safety property.

Our experiment: SIGKILL mid-trajectory resumes byte-identical; non-idempotent commands execute exactly once [MEASURED].

Exchange rate: SCHEDULING buys utilization and unbiased data; spends orchestration complexity.

Implemented [IMPLEMENTED]

  • Sandbox: isolated file namespace with escape prevention; deterministic scratch state.
  • Trajectory log: append-only, hash-chained steps (seq, prev, payload_hash, tool, args, result, hash); tamper-evident (verified by test).
  • Runner: checkpoint after every step (sandbox snapshot + log length); resume restores state and fast-forwards positionally, logged commands are never re-executed, plan/log divergence is refused.

Invariants proven (Exp12 + tests)

invariant result
uninterrupted vs preempted(os._exit(137) at step 4) → resumed byte-identical trajectory hashes ✓
non-idempotent append_counter values after resume [1, 2], no double execution ✓
sandbox path escape PermissionError ✓
log tampering chain verification fails ✓

[MEASURED]

Why regeneration is wrong [PAPER Section 5.2.3]

Re-running a killed trajectory from scratch biases training data toward shorter trajectories (short ones survive interruption more often). Resume from WAL + checkpoint preserves the true length distribution. Our runner demonstrates the toy version: resume continues at step 5 of 6.

Answering the spec question

When agent trajectories become part of learning, is the runtime part of the training system? Yes operationally: crash-consistency and replay semantics determine the data distribution itself: they are training infrastructure, not serving details.

Limitations

  • Single-process toy; no concurrent sandbox pool or layered image loading (DSec's EROFS/overlaybd substrate concerns).
  • Plan must be deterministic for positional fast-forward to apply.