A ctlplne studio product
trstctl /docs GitHub ↗ Live demo

Key custody: where every private key is generated and where it lives

Whose process created the private key, and whose disk holds it, is the question an auditor asks first and the one a whitepaper is worst at answering. This page answers it per credential kind, in one table, and a CI guard (docs/custody_test.go) keeps the table matched to the code.

Read it as a map of the issuance boundary. A requester-supplied CSR or public key does not send its private key to the control plane. That observation cannot prove where the key was originally generated or whether a previous operation handled it. Generated by the control plane means this issuance did handle it, and says so.

Mechanism

A private key can be created in exactly three places in a trstctl deployment:

  • The requester — your host, your workload, your device. A PKCS#10 request travels up, a certificate travels down, and the control plane signs something it cannot read a key from. This is the target shape for every credential kind.
  • The isolated signer — CA keys only. They are created inside the signer process (AN-4), never leave it, and are never subject keys.
  • The control plane — the remaining legacy paths, each named below with what replaces it.

Per certificate and per kind

The table below answers this per credential KIND, which is the right level for a design review and the wrong level for an audit. An auditor is not asking about ACME in general — they are asking about the certificate in front of them, and a kind-level table cannot say whether that one took the modern path or the deprecated server-keygen path retained beside it.

So every certificate now carries its own custody record (epic B5), written from what the issuing code actually did: origin, storage class, exportability, and the actor where one is known. The certificate detail page shows it, and the served API carries it on every certificate response.

For host-agent renewal, issuance initially records the facts known at CSR signing: the key originated on the authenticated host agent and which agent generated it. The control plane does not guess the final storage location before installation. After the connector installs the certificate, the agent's v2 terminal receipt signs the certificate fingerprint, origin, storage class, exportability, and generator with the same key behind its mTLS certificate. Missing, partial, connector-inconsistent, wrong-attempt, or altered custody is refused; the job is not completed. The verified receipt appends certificate.custody.attested, and that event projects the final four fields so replay reconstructs the same inventory.

Empty means unrecorded, and that is a third answer rather than a default. A certificate a network scan found has an origin nobody observed, and the console says so in words — "unrecorded is not the same as safe" — rather than letting a blank field read as reassurance. Custody written at issuance is never overwritten by a later discovery upsert, so a scan re-finding a certificate cannot quietly turn an audit fact back into an unknown.

Framework evidence packs aggregate those tenant-scoped rows. Their signed manifest contains counts by the closed custody vocabulary and explicitly lists every incomplete certificate plus the fields it lacks. The Policy console renders the same gap list. A legacy row with only key_origin is incomplete evidence, not silently counted as a complete custody record.

The table

Credential kind Key source for this issuance Key stored where Does this issuance handle the private key in the control plane?
ACME leaf (RFC 8555) The ACME client Wherever the client put it No
EST leaf (RFC 7030) The EST client Wherever the client put it No
SCEP leaf (RFC 8894) The SCEP client / MDM The device No
CMP leaf (RFC 4210) The CMP client Wherever the client put it No
Identity leaf, subject_csr_pem supplied The requester Wherever the requester put it No
Identity leaf, no CSR, agent-executed target The host agent Installed into the target's connector-specific host store; transient copies stay in locked host memory No — only the CSR travels up
Identity leaf, no CSR, retained control-plane target The control plane Tenant-bound encrypted recovery record; sealed deploy intent when a control-plane connector is configured Yes — deprecated, see below
CA root / intermediate The isolated signer Inside the signer's key store or an HSM No — it never leaves the signer
Agent enrollment identity The agent The agent's own key store No
SSH host / user certificate The requesting host or user Wherever the requester put it No
SPIFFE X.509-SVID, host-agent Workload API The host agent Transient locked memory in the agent, then returned to the workload over that host's UDS No — only the public key travels up
SPIFFE X.509-SVID, control-plane compatibility Workload API The control plane Transient locked memory, then returned over the control-plane-local UDS Yes — deprecated compatibility path, see below
PKI-as-a-secret, csr_pem supplied The requester Wherever the requester generated it No
PKI-as-a-secret, common_name supplied The control plane Returned once in the response, not persisted Yes — deprecated, see below
Automated renewal successor, agent-executed target The host agent Installed into the target's connector-specific host store; transient copies stay in locked host memory No — only the CSR travels up
Automated renewal successor, recorded subject_csr_pem The original requester Wherever the requester kept that matching key; renewal creates no new subject key No
Automated renewal successor, no CSR and retained control-plane target The control plane Tenant-bound encrypted recovery record; sealed deploy intent when a control-plane connector is configured Yes — deprecated identity fallback, see below
Ephemeral workload credential (attested) The attested workload Wherever the workload put it No — it presents its own public key
Direct attested X.509-SVID The requesting workload Not established by the attestation request No — only the public key travels up

ACME, EST simple enrollment, SCEP, CMP, and direct attested issuance record key_origin=requester through certificate.recorded. They leave storage, exportability, and the named key generator unrecorded: receiving a CSR or valid workload proof does not establish those facts. The detail describes this issuance; it does not attest the key's entire history. EST server key generation uses a separate interface and does not acquire requester provenance from this rule. Older rows without custody evidence stay unknown; the console does not invent an explanation for missing evidence or silently backfill a security claim.

The three retained control-plane generators, and what replaces each

There are four Yes rows because the identity generator is reused by its automated-renewal fallback. There are only three distinct control-plane subject-key generators to remove.

Identity leaf without a CSR, including its renewal fallback. Supplying subject_csr_pem is the primary direct-identity path: trstctl signs the request you built and generates nothing. On renewal, the recorded CSR is honoured again, so the requester keeps the same key and the control plane still generates nothing.

If the deployment target says executor=agent, first issuance and renewal take a different branch before any mint. The control plane queues endpoint.renew; the host agent generates the key, sends only a CSR upward, receives the certificate, installs the pair, verifies the live endpoint, and signs the custody receipt. This is the shipped B2 path, not future work.

The retained fallback applies only when there is no CSR and the target is not agent-executed. It calls the same control-plane key generator for first issuance and renewal, and records issuance.server_side_keygen. The key is held in locked memory. Before signing, a tenant-bound encrypted recovery record retains the key and CSR so a retry uses the same subject. Inventory records key_origin=control_plane, key_storage=sealed_store, and key_exportable=exportable. These describe the encrypted recovery material; they do not assert hardware protection. Older records with missing exportability remain unrecorded because no new issuance evidence establishes their history. If a control-plane connector is configured, the bytes are also sealed into its outbox deploy intent; otherwise no deploy material is emitted. Working copies are wiped after the dispatcher finishes. This compatibility path is deprecated, and the event identifies the flows that must move to subject_csr_pem or an agent-executed target.

Lifecycle issuance also retains the exact public certificate template before the CA signature. The isolated signer journals that operation and returns the same signature on retry. An unfinished outbox command keeps its template and encrypted subject preparation beyond the normal idempotency retention window. Once delivery completes, normal retention can reclaim these preparation records. Retained deployment intents have their own lifetime. Historical commands that signed before this mechanism cannot reconstruct a lost certificate or key from a request ID.

If first issuance records the certificate but cannot queue its connector deployment, redelivery resumes deployment with the original encrypted key and recorded public chain. It does not sign another certificate. Recovery refuses missing key preparation, changed issuer or subject binding, mismatched material, and an expired or revoked certificate. A deployment already queued is not queued again by this issuance retry. These checks do not replace verification of the actual endpoint.

Control-plane compatibility Workload API. The production host agent now serves the SPIFFE Workload API on the workload's machine. Its FetchX509SVID generates a locked key before calling upward, sends only the public DER plus locally attested selectors to the control plane, and returns the private half only over that host's Unix-domain socket.

The control plane still composes its older Workload API socket during the deprecation window. A workload that dials that compatibility socket receives a key generated in the control-plane process, so that row remains Yes. The host-agent Workload API is its replacement; the two rows stay separate until the compatibility socket is removed.

PKI-as-a-secret with only a common name. /api/v1/secrets/pki now requires exactly one custody mode. Supply csr_pem and trstctl validates the self-signed PKCS#10 request through the crypto boundary, signs it, and returns only the certificate; the matching key never entered the control plane. Supplying common_name keeps the legacy brain-local convenience: trstctl first commits an issuance.server_side_keygen deprecation receipt, then generates and returns a keypair. If that receipt cannot become durable, issuance fails before key generation. The Secrets console defaults to CSR mode and links the legacy choice to its filtered Audit evidence. Vault/OpenBao clients use pki/sign/{role} for CSR custody; pki/issue/{role} is the same deprecated server-keygen choice.

What this page does not claim

Custody is about the subject key. It says nothing about whether a certificate reached its endpoint, which is a separate question with a separate answer, and nothing about how a requester protects the key once it has it — a key generated on your host and left world-readable is your risk, not a custody boundary trstctl can enforce.

Rendered live from github.com/ctlplne/trstctl — found a mistake? edit this page.