In February 1975, around 150 scientists gathered at the Asilomar Conference Grounds in California, to discuss a technology they themselves were building. The technology was recombinant DNA, the ability to cut genetic material from one organism and splice it into another. It promised enormous advances in medicine and biology, but the scientists feared that engineered organisms could escape labs and create consequences they did not yet understand.

Before Asilomar, Paul Berg and other leading molecular biologists had already called for a voluntary moratorium on some experiments. At Asilomar, they went much further and agreed on safeguards, containment standards and restrictions under which research could continue. Those principles helped shape later NIH guidelines. Biotechnology did not stop, but it became considerably safer for humanity.

A few decades earlier, another group of scientists had faced a similar dilemma, but the outcome was very different.

During the Manhattan Project, Niels Bohr warned that atomic weapons could create an uncontrollable arms race unless countries developed systems of international control. Pioneer Leo Szilard and other scientists urged restraint before the bomb was used. Their warnings went largely unheeded, and the horrors of Hiroshima and Nagasaki followed. Only later did Oppenheimer become the most famous public voice calling for control of the technology he had helped create, and a frightened world got together to create the IAEA and Non-Proliferation Treaty.

It is often said that history does not repeat itself, but it rhymes; in AI we may now be approaching the point where we must decide which historical rhyme we prefer.

Warnings about AI safety are not new. Geoffrey Hinton left Google in 2023 partly so he could speak more freely about AI risks. Yoshua Bengio has repeatedly warned about systems becoming difficult to control. Researchers have left OpenAI and Anthropic complaining that capability development was outrunning safety work.

But over the past week, the warnings have reached something close to a crescendo.

Jacob Coxon, a 27-year-old researcher who worked at both OpenAI and Anthropic, resigned from the industry, accusing frontier labs of “gambling with our lives”. He argued that some people inside these companies genuinely believe advanced AI could pose an existential risk before the end of the decade.

And, then came the potential tipping point: Dario Amodei, CEO of Anthropic, one of the companies racing hardest to build frontier AI, published a remarkable essay titled We Must Pace the Frontier. His argument was not to stop AI, but to slow capability development enough for safety and control mechanisms to catch up. Arch-rivals Sam Altman of OpenAI and Elon Musk publicly supported the broad thrust of his argument, with Altman specifically backing independent evaluators with deep access to frontier systems.

Two developments particularly worry Amodei. The first is recursive self-improvement. AI systems are increasingly helping researchers write code, run experiments and build better AI systems. Better AI helps build better AI, which in turn becomes still more capable of helping build its successor. The fear is not that a machine has suddenly become autonomous, but that this feedback loop could accelerate capability growth beyond our ability to understand or control it. The second is recent evidence that advanced agents can behave in unexpected ways outside tightly controlled environments. In one incident, OpenAI agents breached the Hugging Face software platform during testing. The concern is what similar behaviour could look like when systems become far more capable, a possibility that could become a reality in less than a year.

Amodei proposes three broad responses: independent evaluators embedded inside frontier labs, common safety standards across major companies, and eventually international coordination, including across geopolitical rivals.

The last is by far the hardest.

Every company has an incentive to keep racing if it believes competitors will not slow down. So does every country - especially the US and China. The US worries that restraint will hand China an advantage, and China has equivalent concerns about the US. It is the classic prisoners dilemma and everyone can see the cliff, but nobody wants to brake first.

Some AI boomers are warning that the alarm is exaggerated. Perhaps recursive self-improvement will proceed more slowly than feared, and today’s strange agent behaviour will look primitive and manageable in hindsight. But when the downside being discussed is economic or even species collapse, with mass cyber disruption and loss of human control over the technology, the usual objection that “they may be crying wolf” becomes less persuasive.

Prometheus gave humanity fire. Fire transformed civilisation, but only because we also learnt to contain it in hearths, furnaces and controlled engines. The automobile offers another useful lesson. Brakes were never anti-car. As cars became faster, braking systems became more sophisticated. Better brakes enabled greater speed rather than preventing it. AI safety should be thought of in the same way. The objective is not to extinguish the fire or stop the car. It is to ensure that capability does not outrun control.

Which brings us back to Asilomar.

The scientists there did not wait for a biological disaster before acting. They debated and agreed on safeguards and guardrails while the technology was still young enough to shape. We now know that AI needs something similar.

And perhaps India could help convene the modern Asilomar. India sits in an unusual position: close to the US but not allied to it, a member of BRICS and G20, deeply engaged with the Global South, and committed to building its own AI capabilities without belonging entirely to either technological bloc. It has also tried to position itself as a bridgebuilder in global technology governance, as the February AI Summit and recent BRICS conference have demonstrated.

Could India invite the frontier labs, China, the US, Europe and researchers from around the world to Bangalore or New Delhi to negotiate a basic framework for pacing the AI frontier?

The time is now. History has taught us repeatedly: when the people building a transformative technology begin warning that they may be losing control of it, the rest of society should listen before, not after, the explosion. We can have AI’s Asilomar moment. Or we can wait for its Hiroshima.