The High-Stakes Heist of Trust in AI Development

The recent calls for restraint in artificial intelligence (AI) development are a stark reminder that innovation is outrunning our ability to ensure its safety. Amidst warnings about the dangers of unbridled progress, one crucial factor stands out: trust – or rather, the lack thereof – between the United States and China in AI development.

The notion that these two global powers will ever come to a mutual agreement on the use of AI is unrealistic. As Wagman points out, broad shared principles inevitably collide with geopolitical realities. The challenge lies not in forging an all-encompassing trust but in finding common ground on catastrophe.

The stakes are high: if either side believes the other is secretly advancing while they restrain themselves, the incentive to defect becomes overwhelming. This creates a self-reinforcing cycle of mistrust that undermines even the most well-intentioned safety measures. Instead of negotiating how fast AI should move, nations should focus on when to brake – and what indicators signal an impending need for restraint.

The False Promise of Global Governance

International coordination is often touted as essential for regulating AI development. However, this approach assumes a level of trust and cooperation that may never materialize between the US and China. Moreover, relying solely on global governance risks creating a bureaucratic quagmire where competing interests and agendas stifle progress.

In practice, international agreements are frequently negotiated in a vacuum, with little consideration for how they will be enforced or monitored. This can lead to a situation where nations pay lip service to cooperation while secretly pursuing their own interests. The result is a system that is more focused on appearances than actual results.

A Safety Framework Built on Verification

Rather than trying to negotiate a permanent global speed limit for AI, nations should concentrate on establishing an early warning system that alerts developers when their creations are approaching catastrophic capabilities. This could involve shared indicators of rapid growth in autonomous performance, sophisticated attempts at manipulation, or unusual increases in compute consumption.

Professional evaluations would play a crucial role in this framework, assessing whether agreed-upon warning indicators have been triggered. A broader coalition, comprising governments, financial institutions, and international organizations, would then generate pressure through collective consequences, making non-compliance sufficiently costly to become strategically irrational.

The Power of Collective Consequences

The Financial Action Task Force (FATF) provides a useful precedent for this approach. In the fight against terrorist financing and proliferation, nations often didn’t trust one another but agreed on limited common threats and implemented shared standards. Professional evaluations assessed compliance, generating pressure through governments, markets, and international organizations.

This model is not without its flaws, as perfect verification is impossible in any system that relies on competing interests and motivations. However, the objective should be to make cheating sufficiently detectable and costly that compliance becomes strategically rational – not to aim for an unattainable ideal.

Beyond Governance: A Coalition of Consequences

The international community’s role is not to govern AI by committee but to help make a narrow safety bargain between the two leading powers credible enough to survive. This coalition will not need to decide whether a model is approaching recursive self-improvement; its task is to convert technical determinations into collective consequences.

A Safety Bargain for the Ages

The high-stakes heist of trust in AI development demands a more nuanced approach than simplistic calls for global governance or unilateral restraint. By focusing on early warning indicators, verification, and collective consequences, nations can create a safety framework that works within the harsh realities of geopolitics – not despite them.

In this game of cat and mouse, where the rewards for non-compliance are high and the risks of catastrophic failure are ever-present, it’s time to acknowledge that trust is not a necessary ingredient. What matters most is the ability to make cheating sufficiently costly that compliance becomes the only rational choice. The future of AI development hangs in the balance – and it’s time to get real about what this means for nations, corporations, and individuals alike.