How Pacing the Frontier Fails to Ensure AI Safety

The debate over AI safety has reached a fever pitch, with industry leaders like Dario Amodei advocating for a “pacing” approach to slow down the development of advanced AI systems. On the surface, pacing seems reasonable: it’s akin to deploying a pace car on a racing track to regulate drivers’ speed as they approach hazardous corners. However, as we examine Amodei’s proposal more closely, the metaphor becomes clear: it’s not only misguided but fundamentally flawed.

The idea of pacing the frontier is rooted in Formula 1 racing, where a pace car regulates driver speed when conditions become too hazardous. Applying this concept to AI development oversimplifies the complexities involved. Amodei proposes slowing down progress by introducing third-party evaluators, common safety standards, and limits on unchecked AI growth. While these measures seem prudent, they ultimately fail to address the core issue: ensuring that AI systems are designed with safety in mind from the outset.

Amodei’s letter is notable for its reassuring tone, aimed at appeasing investors and policymakers. He explicitly states that “pacing does not mean halting model training or technical progress.” This claim contradicts his earlier assertion that limits on unchecked AI growth are necessary. It appears Amodei is walking a fine line between slowing down progress and maintaining the momentum of his company’s research.

Setting a slower rate of progress for capabilities while hoping it provides enough time to get safety right is not a viable solution. The safety requirements should be non-negotiable, with further progress contingent upon meeting those requirements. For instance, imagine if Boeing introduced new planes every year without first obtaining airworthiness certification; the public would demand accountability.

Interestingly, Amodei’s own document suggests he agrees with this critique. He proposes rules that require models to meet safety standards before being developed further. This is precisely the “red lines” approach advocated by AI safety researchers: setting clear boundaries for safety and holding developers accountable when those boundaries are crossed.

However, there’s a crucial distinction between Amodei’s proposal and the actual red lines approach. While he proposes setting safety requirements and having developers meet them, his plan still relies on the assumption that progress can continue at a steady pace even as safety concerns are being addressed. This is where the pacing metaphor falls short – it implies that safety can be an afterthought rather than the primary consideration.

The risks associated with uncontrolled AI growth cannot be overstated. Recursive self-improvement leading to superintelligent AI raises the prospect of irreversible loss of human control. The acceptable risk level for such a catastrophic outcome is arguably one in 100 million per year – not the one in 10 or one in five estimated by some industry leaders.

As we navigate this complex landscape, it’s essential to recognize that sunk costs are just past investments that cannot be recouped by continuing down an unsustainable path. The current technology trajectory has already received trillions of dollars in funding, but if we’re truly committed to responsible AI development, we must acknowledge the imperative to fundamentally change course.

The present level of attention to AI risk is welcome, but it’s still a far cry from what’s required. We need industry leaders to take concrete action, not just propose vague solutions that placate investors and policymakers while maintaining the status quo. The future of human civilization depends on our ability to make responsible choices in the development of advanced technologies.

Ultimately, pacing the frontier is a metaphor that has outlived its usefulness. It’s time to set aside comforting illusions about AI safety and confront the harsh realities head-on.