We’ve been thinking about trust wrong.

The human-machine teaming conversation usually frames trust as a calibration problem: how do we get humans to trust AI the right amount? Not too much (automation complacency), not too little (underutilization). Find the sweet spot, train to it, problem solved.

This framing misses something fundamental. Trust isn’t a dial you set once. It’s a dynamic property of a system—one that emerges, propagates, degrades, and recovers across every node in the network.

Trust is a distributed system problem.

The Byzantine Generals of Human-Machine Teams

Distributed systems engineers have spent decades solving trust problems. How do you coordinate action when some nodes might fail? When messages might be delayed or lost? When you can’t verify every participant’s state in real-time?

The solutions aren’t about eliminating uncertainty. They’re about operating despite it.

Human-machine-AI teams face the same fundamental challenges:

  • Partial observability. No single node—human or machine—has complete information about system state.
  • Asynchronous updates. Trust-relevant information (capability changes, performance history, context shifts) propagates at different speeds to different participants.
  • Partition tolerance. Teams must continue functioning when communication degrades, which means trust decisions must be made locally with incomplete data.

The question isn’t “how much should the human trust the AI?” It’s “how does trust flow through this system, and what happens when that flow is interrupted?”

Trust Has a Topology

In any human-machine team, trust isn’t uniformly distributed. It has structure:

Hierarchical trust. A squad leader trusts their autonomous wingmen differently than they trust a remote AI advisor. A field commander trusts aggregated intelligence differently than raw sensor feeds. Authority boundaries create trust boundaries.

Earned trust. A system that has performed reliably in similar conditions accumulates trust capital. A system operating outside its demonstrated envelope should trigger appropriate skepticism—automatically, not just when a human remembers to be skeptical.

Transitive trust. If I trust the commander, and the commander trusts the autonomous system’s assessment, how much of that trust transfers to me? Distributed systems have formal models for this. Human-machine teams mostly wing it.

Trust decay. Trust isn’t static. A system that hasn’t been validated recently, that’s operating in novel conditions, or that’s reported anomalies should see its trust score degrade over time. This isn’t distrust—it’s appropriate uncertainty.

What This Means for Architecture

If trust is a distributed system property, then your coordination architecture needs to treat it as first-class data:

Trust must be observable. Every node should be able to query its own trust relationships and be aware of how it’s trusted by others. Opacity breeds miscalibration.

Trust must propagate. When conditions change—a sensor degrades, an AI model encounters distribution shift, a human operator becomes fatigued—that information needs to flow to nodes whose decisions depend on it.

Trust must be local-first. You can’t phone home to ask whether you should trust your wingman. Trust decisions happen at the edge, which means the edge needs sufficient context to make them well.

Trust boundaries should match authority boundaries. The natural structure of human organizations—squads, platoons, companies—represents evolved solutions to trust coordination. Technical architecture should respect these boundaries rather than flattening them.

The Human in the Loop, Reconsidered

“Human in the loop” is often framed as a constraint—a regulatory or ethical requirement that slows down autonomous action. But viewed through the lens of distributed trust, humans serve a different function: they’re trust anchors.

In a system where machine nodes are making increasingly autonomous decisions, human judgment provides:

  • Ground truth calibration. Humans can recognize when the world has shifted in ways that invalidate machine assumptions.
  • Trust reset authority. When something goes wrong, humans can intervene to re-establish trust baselines.
  • Contextual override. Humans carry context that may not be encoded in machine state—political considerations, ethical boundaries, commander’s intent.

The goal isn’t to keep humans in every loop. It’s to position humans at the right points in the trust topology—where their judgment has maximum leverage and their attention isn’t wasted on decisions machines can reliably make.

Building Trust-Aware Systems

We’re building HIVE with trust as a core architectural concern, not an afterthought:

  • Capability advertisement lets nodes declare what they can do and under what conditions—making trust assessable rather than assumed.
  • Hierarchical aggregation ensures trust decisions happen at appropriate levels, with appropriate context.
  • Graceful degradation means that when trust relationships are disrupted (by communication loss, node failure, or detected anomalies), the system continues operating with appropriately reduced confidence rather than failing catastrophically.

Trust isn’t a human factors problem bolted onto a technical system. It’s a distributed system property that emerges from architecture choices.

Design accordingly.

This is the second in a series on the principles behind HIVE Protocol. Previously: The Device Is The Network