The credential problem hiding inside every agent
An agent that can call a model is interesting. An agent that can deploy code, query customer data, rotate keys, or approve a payment is useful. The second kind also has something the first does not: authority worth stealing.
The familiar answer—put a token in a Kubernetes Secret and expose it as an environment variable—creates the wrong security boundary. Plaintext reaches the API server and etcd. It can leak into process listings, crash reports, debugging output, or an overpowered controller. Rotation becomes a distributed race. A bearer token remains useful to whoever copies it.
The product thesis: identity must travel with authority
Most agent platforms separate identity, authorization, secret management, and workflow orchestration into independent setup tasks. The seams become the vulnerability. A correctly signed token can still carry excessive scope. A properly encrypted secret can still be delivered to the wrong pod. A durable rotation can still preserve plaintext forever in workflow history.
AgentShield treats those controls as one chain. Every handoff answers four questions: who is acting, what authority remains, which live workload receives it, and what durable evidence can be kept safely? That creates a practical control plane for delegated trust rather than another vault or authentication proxy.
The boundary I chose
AgentShield is intentionally not another identity provider. An external OIDC/OAuth system remains the issuer and owns signing keys, JWKS publication, and account recovery. AgentShield is the client-side enforcement layer: it validates tokens strictly, constrains delegated authority, protects refresh tokens, and brokers workload secrets.
This separation kept the prototype focused. I could demonstrate meaningful agent security without quietly inventing a second authorization server or claiming control over provider signing keys.
Plaintext has a place—and it is not etcd
The stored Kubernetes object contains a versioned AES-256-GCM envelope: ciphertext, nonce, a KMS-wrapped data-encryption key, and authenticated metadata. Namespace, secret name, schema version, and generation are bound as additional authenticated data, so copying or editing the envelope breaks decryption.
Ingest locally. The writer generates a fresh DEK, encrypts the value, asks Cloud KMS to wrap the DEK, and submits only the envelope.
Prove workload identity. A sidecar presents a short-lived, audience-bound ServiceAccount token. The broker calls TokenReview and resolves the live pod instead of trusting caller-supplied labels.
Authorize before decrypting. Namespace, ServiceAccount, pod UID, and selectors must satisfy the AgentSecret policy. The default is deny.
Deliver into memory. The broker unwraps and decrypts in process memory. The sidecar atomically writes a mode-0400 file into a memory-backed emptyDir.
This is a bounded claim, not magic. Plaintext exists briefly in process memory and in the pod’s tmpfs. A privileged node or cluster administrator can still read it, and Python cannot promise perfect zeroization. What the design removes is plaintext from Kubernetes API objects and ordinary persistent-volume paths.
Identity is more than “the JWT decoded”
The identity layer pins issuer and audience, allows only configured asymmetric algorithms, rejects attacker-controlled key URLs, bounds discovery and JWKS responses, refreshes once for an unknown kid, and fails closed when required state is unavailable.
Delegation uses an external RFC 8693 endpoint. Before exchange, requested scopes must be the intersection of the subject’s authority and the agent’s allowlist. After exchange, AgentShield checks the returned issuer, audience, subject relationship, scopes, expiry, and proof binding. It never widens authority and never mints a substitute token.
For high-sensitivity operations—writes, administration, payment, identity, token, credential, and secret management—the prototype implements DPoP first. The proof binds the method, normalized URI, timestamp, nonce where required, token hash, and a single-use identifier stored in a replay cache. There is no silent bearer fallback.
Where Temporal belongs
Rotation crosses slow and failure-prone systems: create a provider credential, encrypt it, publish a generation, verify delivery, promote it, and revoke its predecessor. That is exactly where durable workflow semantics help.
Temporal does not sit in the request path. Workflow history is durable by design, so plaintext, bearer tokens, private keys, and raw provider errors are forbidden from workflow inputs, results, memo, search attributes, and exceptions. OCI tests cover retry, compensation, idempotency, replay, and a canary scan of history.
A product architecture, not a bag of controls
The value is not any individual standard. OIDC answers identity. RFC 8693 narrows delegated authority. DPoP makes selected tokens harder to replay. Kubernetes identifies the live workload. KMS protects the key hierarchy. Temporal makes multi-system rotation recoverable. AgentShield composes them into one enforceable lifecycle.
| Plane | Responsibility | Failure posture |
|---|---|---|
| Identity | Validate issuer, audience, signature, state, nonce, and proof binding | Reject ambiguous or unverifiable identity |
| Authority | Intersect scopes, constrain audience, cap lifetime, preserve delegation lineage | Never mint or widen authority locally |
| Delivery | Resolve the live pod, authorize it, unwrap only after approval, write to RAM | Deny before decryption |
| Lifecycle | Coordinate rotation, verification, promotion, revocation, and compensation | Keep the previous generation active until the candidate is proven |
This composition also clarifies ownership. Identity providers remain responsible for token issuance and signing keys. Cloud KMS remains responsible for root key protection. Kubernetes remains responsible for workload identity and scheduling. Temporal remains responsible for durable coordination. AgentShield owns the policy and the secure transitions between them.
What was actually demonstrated
| Claim | Evidence | Boundary |
|---|---|---|
| Kubernetes identity | Broker received its intended GKE Workload Identity | Single-node validation cluster |
| KMS envelope path | Generation 1 encrypted; authorized retrieval matched its independent digest | Data Access audit events were not available |
| Authorization | Allowed workload succeeded; different ServiceAccount/label returned 403 | Live binding-removal test remains future work |
| Memory delivery | Explicit sidecar and RAM-backed emptyDir | Privileged node access is outside the guarantee |
| Durable rotation | Retry, compensation, idempotency, replay, and history redaction passed on OCI | No production Temporal service deployed |
| Supply-chain checks | 46 tests plus lint, types, SAST, dependency and container scans | Prototype, not certification |
What changes operationally
AgentShield shifts secret access from deployment-time possession to request-time authorization. Applications no longer need a durable credential injected into their environment. Platform teams gain one policy point that understands both cryptographic identity and live workload context.
It also separates fast and slow decisions. Token validation and secret authorization stay synchronous and fail closed. Credential rotation becomes a durable workflow with retries and compensation. This keeps Temporal out of latency-sensitive requests while using it where partial progress and recovery actually matter.
Where this becomes useful
| Environment | Agent action | AgentShield boundary |
|---|---|---|
| Software delivery | Deploy, roll back, or read CI credentials | Short-lived identity, scoped delegation, audited secret retrieval |
| Data platforms | Query governed warehouses or rotate connectors | Audience/scope limits and revocable credential generations |
| Customer support | Inspect tenant-specific operational data | Pod-bound identity and policy tied to workload context |
| Financial operations | Initiate sensitive write or payment workflows | DPoP required; no bearer fallback for high-sensitivity routes |
| Long-running agents | Survive credential rotation and partial outages | Temporal compensation without secrets in workflow history |
The broader idea is simple: agent identity should be a shrinking chain of authority. Each hop should be narrower, shorter-lived, bound to its caller where practical, and independently observable.
Three implications for agent platforms
- Identity without delegation policy is incomplete. Knowing which agent is calling does not prove that the requested audience, scope, lifetime, or downstream action is appropriate.
- Secret managers need a delivery architecture. Encryption at rest says little about the final mile. The decisive controls live between workload attestation, authorization, decryption, and the application’s read boundary.
- Durability creates its own data boundary. Workflow engines are excellent for recovery, but their histories are databases. Secret-safe orchestration means passing references and receipts, not sensitive payloads.
What production would still demand
This is a serious prototype, not a production claim. The next layer includes multi-cluster availability, managed Redis and Temporal operations, immutable audit export, KMS Data Access logging, load and chaos testing, live revocation and pod-recreation exercises, upgrade guarantees, additional KMS providers, and formal compliance work.
The design also deliberately defers a CSI driver, automatic sidecar injection, mTLS-bound OAuth tokens, and an AgentShield-owned issuer. Those could be useful later; none is necessary to prove the core boundary.
What to check in review
- Can unverified claims influence a network destination, key choice, or authorization decision?
- Can requested scopes become a union instead of an intersection?
- Does plaintext enter a Kubernetes object, environment variable, log, trace, or workflow history?
- Does a missing replay store, JWKS key, TokenReview response, or KMS dependency fail closed?
- Are writer and reader identities separated, and are selectors resolved from the live pod?
- Can cleanup run after every error path, including partial rotation and failed promotion?
The lesson
AgentShield is the smallest system I could build that demonstrates those properties together—and records the places where the proof is not finished.
Explore the source Read the evidence