Federating Riptides with SPIRE

A trust domain only takes you so far. Inside one, SPIFFE answers the question of who a workload is and every peer shares the same issuing authority, which is the easy case. The interesting question is what happens when two independently operated trust domains have to talk to each other, and the answer is federation.

We have written about the mechanism twice: once about what federation is and the bundles and endpoints it is built from, and once about the argument that federation is the easy part while operating the resulting distributed system is the hard part. What we have not done is build one and show the commands.

For the other side of that federation we are using SPIRE, because SPIRE is what most people mean when they say they are running SPIFFE. It is the reference implementation, it is the one the specification is usually read alongside, and if there is a SPIFFE deployment somewhere in an organisation today then it is probably this one. That makes it the most useful counterpart to demonstrate against: nothing about the setup below is specific to Riptides talking to Riptides, and both ends are doing what the specification says rather than anything private to either implementation.

So this post is the whole exercise end to end, with every command: a stock SPIRE deployment publishing a bundle endpoint on the public internet, a Riptides deployment alongside it, bundle exchange in both directions, a workload in each domain completing mutual TLS across the boundary, and finally an access policy written against a SPIFFE ID belonging to the other domain. The interesting part is how little of it there is.

Why federate instead of issuing a key

The reason to care about any of this is that the usual alternative is worse than it looks. Two systems under different administrative control that need to talk to each other tend to end up with a credential minted for the occasion: an API key, a service account token, a secret in a vault that both sides can reach. It works immediately, which is most of its appeal, and then it has to be delivered to the caller, stored on both sides, rotated on a schedule nobody enjoys, revoked when somebody leaves, and audited by correlating logs that were never designed to be correlated. It is also usually a bearer credential, which means it proves nothing about who is presenting it beyond the fact that they have it. Everything that goes wrong with long lived secrets goes wrong here, and it goes wrong at exactly the boundary where two organisations can least easily coordinate a fix.

Federation replaces the credential with a verification path. Nothing is minted for the relationship and nothing is distributed, because each side already issues identities to its own workloads and federation only teaches one side how to validate the other’s. There is nothing to rotate, since both sides keep rotating on the schedules they already had, and nothing to leak, because no secret crosses the boundary at any point. What crosses is a set of public keys published at a URL.

Where a trust relationship can be federated, it should be. A purpose built key is the fallback for the cases that cannot, not the default for the cases nobody examined.

One clarification, because federation is usually discussed as though it were a JWT story. It is not. A SPIFFE trust bundle carries both kinds of material, the public keys that validate JWT-SVIDs and the X.509 roots that validate X509-SVIDs, and either can be federated. Every connection in this post is authenticated with X.509, and the distinction is not academic: a JWT is a bearer token, so whoever holds it can present it, while an X509-SVID is proved during the handshake by possession of a private key that never moves. The identity ends up bound to the connection rather than carried inside a request on it.

If the reason to federate is to stop handing out credentials that can be copied, then federating only the token half leaves most of the problem exactly where it was.

What we are building

	SPIRE domain	Riptides domain
Trust domain	`spire.roadrunner.acme.corp`	`riptides.coyote.acme.corp`
Managed by	stock upstream SPIRE 1.15.3	Riptides control plane
Bundle endpoint	`https://203.0.113.10.nip.io`	`https://riptides.coyote.acme.corp/federation`
Profile	`https_web`	`https_web`
Workload	demo server, go-spiffe, terminates mTLS itself	demo client, mTLS established in the kernel

A trust domain is an administrative boundary with a single issuing authority, and an SVID is only meaningful to somebody who trusts that authority. Federation does not dissolve that boundary; it lets one domain learn another domain’s roots so a workload can validate a peer certificate issued by an authority it does not itself use. There is no shared CA, no shared control plane, and no issuance dependency. Each side keeps its own CA, its own rotation schedule, and its own policy, and if one control plane goes down the other keeps serving.

One thing worth stating before we start, because federation invites the wrong assumption in both directions. Riptides is an independent implementation. It is not repackaged SPIRE, it is not built on SPIRE, and it is not a drop-in replacement in the sense of consuming SPIRE’s configuration. It provides what SPIRE provides within the SPIFFE domain and a good deal beyond it: issuance and rotation, mutual TLS established in the kernel rather than in the application or a sidecar, SPIFFE ID based access policy enforced at runtime, and quantum safe key exchange. Federation is the seam where the two meet as equals, and it works because both implement the same specification, not because either knows anything about the other.

Prerequisites

Two Kubernetes clusters, one per trust domain. Ours were GKE 1.35, three nodes each, in the same project.

A Riptides control plane. Free accounts are available on request at console.riptides.io and are fully functional for daemons, workload identities, enforcement, and federation. Registration returns a control plane URL and a trust domain of your own, which are the two values every Riptides command below needs. Point the CLI at it once:

riptides-cli context add --url https://riptides.coyote.acme.corp
riptides-cli context status

For the SPIRE cluster, the usual Helm repositories:

helm repo add spiffe https://spiffe.github.io/helm-charts-hardened/
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo add jetstack https://charts.jetstack.io
helm repo update

Step 1: a publicly reachable SPIRE bundle endpoint

Federation starts with each side publishing its roots somewhere the other can fetch them. The specification defines two profiles. The https_spiffe profile authenticates the endpoint with an SVID, which is elegant but requires the peer to bootstrap trust before it can fetch anything. The https_web profile authenticates it with an ordinary web PKI certificate, so any client can fetch the bundle with nothing pre-shared. We use https_web on both sides.

That choice has a consequence on the SPIRE side. With https_web, SPIRE terminates TLS on the bundle endpoint itself, reading a certificate and key from a mounted secret and reloading them periodically. SPIRE has no ACME client, so something else has to obtain and renew that certificate. cert-manager does, which also means we need somewhere to answer the ACME challenge.

We use a nip.io hostname to avoid owning a domain for the demo, and nip.io resolves any <ip>.nip.io name to that IP. Because the hostname therefore encodes one specific address, the HTTP-01 challenge has to be answerable on port 80 at that same address, and a single L4 service cannot send port 80 to the challenge solver while sending 443 to SPIRE. So an ingress controller goes in first, and its address determines the hostname.

Note the enable-ssl-passthrough flag. The SPIRE chart annotates its federation ingress for passthrough, meaning nginx routes by SNI and never decrypts, but that annotation is inert unless the controller is started with the flag, which is off by default:

helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx --create-namespace \
  --set controller.service.type=LoadBalancer \
  --set controller.extraArgs.enable-ssl-passthrough=true \
  --wait --timeout 7m

kubectl -n ingress-nginx get svc ingress-nginx-controller

NAME                       TYPE           EXTERNAL-IP    PORT(S)
ingress-nginx-controller   LoadBalancer   203.0.113.10   80:30750/TCP,443:30901/TCP

That address gives us the hostname 203.0.113.10.nip.io. Then cert-manager:

helm upgrade --install cert-manager jetstack/cert-manager \
  --namespace cert-manager --create-namespace \
  --set crds.enabled=true \
  --wait --timeout 7m

Now SPIRE itself. Three values matter more than the rest. Setting federation.tls.spire.enabled to false is what selects https_web over https_spiffe; the chart renders one profile or the other, never both. The chart’s default ACME solver is http01: {ingress: {}} with no class, which will not bind to nginx unless the class is pinned. And the issuer template fails the install outright if no ACME email is set:

# spire-values.yaml
global:
  spire:
    clusterName: fed-demo-spire
    trustDomain: spire.roadrunner.acme.corp
    namespaces:
      create: true

spire-server:
  federation:
    enabled: true
    tls:
      spire:
        enabled: false
      certManager:
        enabled: true
        issuer:
          create: true
          acme:
            email: ops@acme.corp
            solvers:
              - http01:
                  ingress:
                    ingressClassName: nginx
    ingress:
      enabled: true
      className: nginx
      controllerType: ingress-nginx
      host: 203.0.113.10.nip.io

helm upgrade --install spire-crds spiffe/spire-crds \
  --version 0.6.1 --namespace spire-mgmt --create-namespace --wait

helm upgrade --install spire spiffe/spire \
  --version 0.30.2 --namespace spire-mgmt \
  -f spire-values.yaml --wait --timeout 8m

The certificate should report ready, which means the challenge succeeded and the secret holds a Let’s Encrypt certificate that SPIRE will pick up from its mount:

kubectl get certificate -A

NAMESPACE    NAME               READY   SECRET                         AGE
spire-mgmt   spire-server-fed   True    spire-server-federation-cert   4m54s

Now the endpoint can be verified from anywhere, with no -k and nothing pre-shared, which is the entire point of https_web:

echo | openssl s_client -connect 203.0.113.10.nip.io:443 \
  -servername 203.0.113.10.nip.io 2>/dev/null \
  | openssl x509 -noout -subject -issuer

subject=CN=203.0.113.10.nip.io
issuer=C=US, O=Let's Encrypt, CN=YR1

curl -s https://203.0.113.10.nip.io/

{
  "keys": [
    { "use": "x509-svid", "kty": "RSA", "x5c": ["MIIDxTCC..."] },
    { "use": "jwt-svid",  "kty": "RSA", "kid": "GWd0l2Rt1Gwey4Szct8omQYeaAcRH3tA" }
  ],
  "spiffe_sequence": 1,
  "spiffe_refresh_hint": 300
}

The bundle is a JWKS carrying both kinds of material at once, the X.509 roots that validate X509-SVIDs and the keys that validate JWT-SVIDs, which is the point made earlier showing up in the payload: the endpoint publishes both whether or not a given peer intends to use both. Everything that follows uses the X.509 half. The refresh hint tells peers how often to return, and the sequence number increments whenever the contents change, so a peer can tell whether anything actually moved.

The Riptides side needs none of this. Its control plane already serves its bundle at /federation on a publicly trusted certificate, so the equivalent step is to fetch it and confirm what is there:

curl -s https://riptides.coyote.acme.corp/federation

Worth noticing that the two domains do not resemble each other internally. SPIRE is signing with RSA here and Riptides with EC P-384, the CA lifetimes differ, and the refresh hints differ by two orders of magnitude. None of that has to be reconciled, which is rather the point of federating instead of merging.

Step 2: exchanging bundles in both directions

Riptides models a federated peer as a FederatedTrustDomain, which takes the peer’s endpoint and polls it on a schedule:

apiVersion: core.riptides.io/v1alpha1
kind: FederatedTrustDomain
metadata:
  name: spire-roadrunner
  namespace: riptides-system
spec:
  trustDomain: spire.roadrunner.acme.corp
  bundleEndpoint:
    url: https://203.0.113.10.nip.io
    profile: HTTPS_WEB

riptides-cli ctl apply -f ftd-spire-roadrunner.yaml
riptides-cli ctl get federatedtrustdomains -n riptides-system

NAME               TRUST DOMAIN                 LAST FETCHED
spire-roadrunner   spire.roadrunner.acme.corp   20s

The status carries a summary of what was retrieved, including certificate serials and key identifiers, which is what makes the relationship verifiable rather than merely configured. Comparing those against what the SPIRE endpoint publishes confirms nothing was copied by hand.

The other direction is a ClusterFederatedTrustDomain, handled by the SPIRE controller manager:

apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterFederatedTrustDomain
metadata:
  name: riptides
spec:
  className: spire-mgmt-spire
  trustDomain: riptides.coyote.acme.corp
  bundleEndpointURL: https://riptides.coyote.acme.corp/federation
  bundleEndpointProfile:
    type: https_web

kubectl apply -f cftd-riptides.yaml
kubectl -n spire-mgmt exec spire-server-0 -c spire-server -- \
  /opt/spire/bin/spire-server bundle list

****************************************
* riptides.coyote.acme.corp
****************************************
-----BEGIN CERTIFICATE-----
...

The className is required rather than decorative. The chart configures the controller manager with a class name and watchClassless: false, so an object without a matching class is accepted by the API server, logged by the admission webhook, and then never reconciled: no status, no error, and an empty bundle list. Confirm the expected value before writing the manifest:

kubectl -n spire-mgmt get cm spire-controller-manager \
  -o jsonpath='{.data.controller-manager-config\.yaml}' | grep -E "className|watchClassless"

Both control planes now hold the other’s roots, fetched over web PKI, each on its own schedule.

Step 3: the workload in the SPIRE domain

Federating the control planes is not by itself enough for a workload to use the federated bundle.

A SPIRE agent only hands a workload the bundles for trust domains named in that workload’s registration entry. The chart’s default ClusterSPIFFEID is a catch-all with no federatesWith, so the entries it generates carry an empty list, the workload never receives the foreign roots, and the handshake fails with an unknown authority error while both control planes look perfectly federated.

So the server gets its own ClusterSPIFFEID:

apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterSPIFFEID
metadata:
  name: demo-server
spec:
  className: spire-mgmt-spire
  spiffeIDTemplate: "spiffe://{{ .TrustDomain }}/ns/{{ .PodMeta.Namespace }}/sa/{{ .PodSpec.ServiceAccountName }}"
  podSelector:
    matchLabels:
      app.kubernetes.io/component: server
  namespaceSelector:
    matchLabels:
      kubernetes.io/metadata.name: demo
  federatesWith:
    - riptides.coyote.acme.corp

The workload is the server half of a small demo application. It is an ordinary SPIFFE application: it fetches an X509-SVID from the Workload API using go-spiffe and terminates mutual TLS itself, with no Riptides code anywhere in it. Because the two clusters are separate, it is exposed through a load balancer so the client can reach it:

kubectl create namespace demo
kubectl apply -f csid-demo-server.yaml

helm upgrade --install riptides-demo \
  oci://ghcr.io/riptides-packages/helm/riptides-demo --version 0.1.0 \
  --namespace demo \
  --set server.spire.enabled=true \
  --set client.enabled=false \
  --set server.service.type=LoadBalancer

spire: serving mTLS as spiffe://spire.roadrunner.acme.corp/ns/demo/sa/default
spire: terminating on :8088, backend on 127.0.0.1:46867
health listening on :8089 (plaintext)

The separate plaintext health listener exists because a kubelet probe cannot present an SVID and therefore cannot use the mutual TLS port. It is worth noticing as a cost of terminating mTLS inside the application, since the Riptides side needs no equivalent: readiness and liveness ports and paths are detected from the pod spec and treated as permissive automatically.

Confirm the registration entry actually federates, because this is the line that is empty when the handshake later fails for no visible reason:

spire-server entry show -spiffeID spiffe://spire.roadrunner.acme.corp/ns/demo/sa/default

SPIFFE ID     : spiffe://spire.roadrunner.acme.corp/ns/demo/sa/default
FederatesWith : riptides.coyote.acme.corp

The server now presents its SVID to anyone who connects, and refuses anyone who cannot present one of their own:

echo | openssl s_client -connect server.203.0.113.20.nip.io:8088 \
  -servername server.203.0.113.20.nip.io 2>/dev/null \
  | openssl x509 -noout -ext subjectAltName

X509v3 Subject Alternative Name:
    URI:spiffe://spire.roadrunner.acme.corp/ns/demo/sa/default

The hostname is only for reachability. SPIFFE authentication is based on the URI SAN, not on DNS.

Step 4: the workload in the Riptides domain

The Riptides side needs a daemon on each node. These are GKE nodes, so GCPIIT attestation is the natural choice: every instance can request a signed identity token from the GCP metadata server, which means no shared secret to distribute and, unlike a join token, something that works for a DaemonSet where every pod authenticates independently.

The Verifier goes on the control plane once, scoped to the project so instances from anywhere else cannot register:

apiVersion: auth.riptides.io/v1alpha1
kind: Verifier
metadata:
  name: gcpiit
  namespace: riptides-system
spec:
  GCPIIT:
    audience: gcp-iit
  requiredMetadata:
    - gcpiit:project:id: "acme-corp-demo"

riptides-cli ctl apply -f verifier-gcpiit.yaml
riptides-cli ctl get verifiers -n riptides-system

Then the daemon, on the second cluster:

helm upgrade --install daemon oci://ghcr.io/riptides-packages/helm/daemon:0.2.0 \
  --create-namespace -n riptides-system \
  --set image.tag="v0.7.0" \
  --set driverLoader.image.tag="v0.5.16" \
  --set driverLoader.driverVersion="v0.7.0" \
  --set config.daemon.controlPlane.url=https://riptides.coyote.acme.corp \
  --set config.daemon.controlPlane.authPlugin.type=GCPIIT \
  --set config.daemon.dataDir=/data/riptides \
  --set config.daemon.trustDomain=riptides.coyote.acme.corp

The daemon loads a kernel module, so the loader first resolves a driver package matching the node’s kernel and distribution. Prebuilt packages are published for a wide range of them, and when the node’s kernel is among them this is an ordinary download and the pod starts in seconds. When it is not, the loader schedules a build automatically instead of failing, and the pod stays in Init:0/1 while that runs:

Determined package name: riptides-driver-cos-19216.532.123_v0.7.0_amd64.tar.gz
Driver package not found (404): ...
Queuing driver build for version v0.7.0 on cos/x86_64...
Waiting for build 8e083676b9887bf8 to complete (timeout: 1800s)...

The build takes a few minutes, after which the module loads and the daemon connects. The result is then published like any other package, so further nodes on that kernel download it rather than triggering a second build: the cost is one-time per kernel version, and only for kernels not already covered. Once the daemons are up:

riptides-cli ctl get daemons -n riptides-system

NAME                                   WORKLOAD ID
aabb9993-e377-41f6-9f08-c09548c4136a   riptides/daemon/acme-corp-demo/europe-west4-b/3047239364927921812
ed80c06e-95ee-4268-8ce5-141fef2baf53   riptides/daemon/acme-corp-demo/europe-west4-b/4860869823415819924
c858f3b9-669f-4536-b853-ed4c47fa5bd7   riptides/daemon/acme-corp-demo/europe-west4-b/8930064552819531412

A workload identity is scoped to a daemon or a group of them, so the group comes first. Selecting on the GCP project means daemons attested from that project join automatically, and new nodes are covered without touching the group. Note the workloadID has to start with daemongroup/:

apiVersion: core.riptides.io/v1alpha1
kind: DaemonGroup
metadata:
  name: fed-demo-riptides
  namespace: riptides-system
spec:
  workloadID: daemongroup/fed-demo-riptides
  selectors:
    - gcpiit:project:id: "acme-corp-demo"

The destination is declared as a Service. The SPIRE side server is an internal service in the sense that matters here: it participates in SPIFFE and does mutual TLS with identities on both sides, as opposed to an external API that would need credentials injected into it.

apiVersion: core.riptides.io/v1alpha1
kind: Service
metadata:
  name: demo-server-spire
  namespace: riptides-system
spec:
  external: false
  addresses:
    - address: server.203.0.113.20.nip.io
      port: 8088
  labels:
    app: demo-server
    trust-domain: spire.roadrunner.acme.corp

And then the client’s identity, which is where the federation finally shows up as policy:

apiVersion: core.riptides.io/v1alpha1
kind: WorkloadIdentity
metadata:
  name: demo-client
  namespace: riptides-system
spec:
  workloadID: demo/client
  scope:
    daemonGroup:
      id: daemongroup/fed-demo-riptides
  selectors:
    - k8s:pod:namespace: demo
      k8s:label:app.kubernetes.io/component: client
  connection:
    tls:
      mode: MUTUAL
      intercept: true
  ingress:
    - port: 3000
      connection:
        tls:
          mode: PERMISSIVE
  allowedSPIFFEIDs:
    outbound:
      - spiffe://spire.roadrunner.acme.corp/ns/demo/sa/default

Three things in there are worth pausing on.

The scope takes the daemon group’s workloadID, not its resource name and not its Kubernetes uid. Both of those are rejected.

The allowedSPIFFEIDs.outbound entry names an identity in the other trust domain. That is the whole payoff of federating: policy on this side is written in terms of the peer’s identity rather than its address.

The ingress override exists for the browser, not for Kubernetes. The workload’s default inbound policy is MUTUAL, which is right for a service and wrong for a browser facing port, since a browser has no SVID to present. Probes are handled automatically, so the only port that needs relaxing is the UI. Writing mode: PERMISSIVE explicitly is worth the extra lines, because an ingress entry with no connection block leaves the mode unset, which silently defaults to permissive: it works, but nothing in the manifest says the port is unauthenticated.

Deploy the client against the address of the server in the other cluster:

helm upgrade --install riptides-demo \
  oci://ghcr.io/riptides-packages/helm/riptides-demo --version 0.1.0 \
  --namespace demo --create-namespace \
  --set server.enabled=false \
  --set client.serverUrl=http://server.203.0.113.20.nip.io:8088

The daemon confirms the match, and the identity it issues is the one the policy above names:

allowed:true  interceptTls:true  mtls:STRICT
primaryWorkloadId:spiffe://riptides.coyote.acme.corp/demo/client

The handshake

kubectl -n demo port-forward svc/riptides-demo-client 3000:3000
curl -s http://localhost:3000/api/check

{
  "ok": true,
  "server": {
    "tls": {
      "type": "spire-x509-svid",
      "spiffe_id":      "spiffe://spire.roadrunner.acme.corp/ns/demo/sa/default",
      "peer_spiffe_id": "spiffe://riptides.coyote.acme.corp/demo/client"
    }
  },
  "client": {
    "type": "STRICT",
    "spiffe_id":      "spiffe://riptides.coyote.acme.corp/demo/client",
    "peer_spiffe_id": "spiffe://spire.roadrunner.acme.corp/ns/demo/sa/default"
  }
}

Each side names the other and the two cross-reference exactly. The SPIRE side validated a Riptides issued SVID from a foreign trust domain, and the Riptides side validated a SPIRE issued one, with neither control plane involved in the other’s issuance. The differing type values are the detail worth keeping: two independent implementations of the same specification, arriving at one connection. The client never speaks TLS, because the kernel module establishes the connection underneath it, while the server is a normal go-spiffe application doing it the usual way.

Who actually enforces any of this

That last sentence is worth unpacking, because it is the substantive difference between the two sides and it is easy to miss when both ends report success.

What federation delivers on the SPIRE side is trust material. The agent hands the workload an SVID, keeps it fresh, and includes the federated roots once the registration entry asks for them. What it does not do, and does not claim to do, is use any of it. Turning that material into a verified connection is the application’s job: open a source against the Workload API, keep it open because SVIDs are short lived, build a TLS config from it, choose an authorizer, check the peer’s SPIFFE ID against something meaningful, and decide what happens when the check fails. The demo server in this post does exactly that, and it is not much code, but it is code, and it exists in every service, in whatever language that service happens to be written in. It also has to keep being correct as that service changes hands.

The failure mode is not that this is hard. It is that the failure is silent and looks like success. An authorizer that accepts any peer chaining to a held bundle is one function call away from one that checks identity, and both produce a working connection with a green log line. The same is true of a verification step skipped under deadline, or an SVID fetched once at startup and never refreshed. The security property depends on every application getting it right, and nothing outside the application can tell whether it did.

Federation distributes the material for a verified connection. It does not make anything verify. On one side of this demo that gap is filled by application code, and on the other it is filled by the kernel.

The Riptides client is the same picture with the enforcement moved. The workload opens an ordinary connection to an ordinary address, and the identity, the handshake, the peer check and the policy are applied beneath it. There is no SPIFFE library linked into the client, no language binding to maintain, and nothing for the application to get wrong, because the application was never asked. That is also why the key exchange can be quantum safe and the policy can be enforced in the same place for every workload: none of it depends on each service adopting a library and keeping current with it. It works the same for a Go service, a Node process, and a binary nobody has the source to.

At a federation boundary this matters more than inside a single domain. The peer is outside your administrative control, the identities come from a CA you do not run, and a permissive authorizer written years ago in one service is now the thing standing between another organisation’s workloads and yours. Moving that decision out of the applications and into one enforced place is not a convenience. It is the difference between a guarantee you can state and a guarantee you have to audit for, service by service.

Policy against a foreign identity

Federation establishes who the peer is. It says nothing about whether the peer should be allowed in, and conflating the two is the most common way federated environments end up more permissive than their operators believe.

Changing one line, to an identity that does not match, while leaving both bundles and the reachable server exactly as they were:

riptides-cli ctl patch wid demo-client -n riptides-system --type=merge \
  -p '{"spec":{"allowedSPIFFEIDs":{"outbound":["spiffe://spire.roadrunner.acme.corp/ns/demo/sa/not-this-one"]}}}'

{"ok": false, "error": "write EPERM"}

The shape of that failure is worth dwelling on. The application did not receive a TLS alert or an HTTP status. It received EPERM on a write syscall, because the connection was refused in the kernel and never left the host. There is no sidecar to bypass and nothing in user space to misconfigure. It is also useful to know before debugging one of these for the first time, since the instinct on a failed mutual TLS connection is to go and read certificates, and the answer here is not in the certificates.

Restoring the original identity restores the connection.

What this does not give you

Two federated domains are still two domains. Policy remains per domain, and there is no unified audit trail spanning both: each side logs its own decisions about its own workloads.

The relationship also depends on continued refresh, which is the failure mode most worth instrumenting, because a federation that quietly stops fetching keeps working perfectly until the peer rotates its CA and then stops all at once. With CA lifetimes measured in hours, as they were here, that window is short. The lastFetched and error fields exist for exactly this, and they belong in monitoring rather than in a setup checklist, which is the same conclusion we reached from a different direction when writing about runtime enforcement.

Where this leaves things

Strip out the explanations and the federation itself is four manifests: one FederatedTrustDomain, one ClusterFederatedTrustDomain, a ClusterSPIFFEID naming the peer domain, and a WorkloadIdentity naming the peer’s SPIFFE ID. Everything else above is scaffolding that any two clusters would need anyway, and the single longest step, getting a publicly trusted certificate onto the SPIRE bundle endpoint, has nothing to do with either implementation.

That is the result worth taking away. Two independently operated trust domains, each with its own CA, its own key algorithm, its own rotation schedule and its own refresh interval, reached mutual authentication by exchanging bundles over a documented profile. The SPIRE deployment was stock throughout, and the application on that side was a normal go-spiffe application that has never heard of Riptides. Neither end needed to know anything about the other beyond a URL and a trust domain name.

Worth noting what never appeared anywhere in the exercise: a credential created for the occasion. No API key was minted for the two domains to talk, nothing was copied from one cluster into the other, and there is no shared secret now sitting in two places waiting to be rotated. The workloads on both sides carry certificates that were already rotating on their own schedules before federation existed, and the only thing that crossed the boundary was a set of public keys fetched over HTTPS.

That matters beyond any single deployment, because cross organizational boundaries are where SPIFFE has to work if it is going to become common, and they are exactly the boundaries that cannot be resolved by putting everyone on one control plane. The direction the specification is moving, which we covered when the SPIFFE roadmap was published, points the same way: toward workloads that cross organizations, providers, and increasingly agent boundaries, where the assumption of a single administrative domain stopped holding some time ago.

Trying this yourself

Notice what the walkthrough never required: a decision about migrating. That is the practical consequence of federation being a boundary mechanism rather than an integration. A Riptides trust domain can stand alongside an existing SPIRE one, each validates the other, and the existing deployment keeps running exactly as it did. Adoption can start with a single cluster, or a single workload, and nothing that already works has to be touched to find out whether the rest is worth having.

What the new domain gives you is not the same capability set in different packaging. Mutual TLS is established without the application taking part, which means it also holds for the services nobody is going to retrofit with a SPIFFE library, and for the ones whose source nobody has. The key exchange is quantum safe by default. Policy is written in terms of SPIFFE IDs and enforced at runtime, including against identities from a federated domain, as the EPERM above showed. The same model covers Kubernetes, virtual machines and bare metal, so the trust domain is not bounded by one platform. Credentials for external services are injected beneath the workload rather than distributed to it, which removes the other half of the long lived secret problem that federation solves for service to service traffic.

The cost of finding out is low. Free accounts are fully functional, federation included, and registration returns a real control plane and a trust domain of your own within minutes. Start Free, point a daemon at it, and everything above is an afternoon’s work. If SPIRE is already running in your environment and you would rather talk through what federating with it looks like in your particular setup before building anything, talk to us.

Free Resource

How exposed are your workloads?

Run the NHI Security Audit Checklist, 8 questions to map your credential exposure, attribution gaps, and lateral movement surface across your own environment. Takes about 15 minutes.

Run the Checklist

spiffe spire federation identity kubernetes