India’s proposed artificial intelligence law may bring some of the world’s most powerful AI models under a separate safety framework, potentially requiring developers to demonstrate that adequate safeguards are in place before releasing systems capable of causing significant harm. The approach would mark a shift from India’s largely incremental, technology-neutral regulation towards closer scrutiny of advanced models, drawing lessons from debates in the United States and elsewhere, NDTV has learnt.
Officials familiar with the discussions told NDTV the government is examining how to address risks arising from increasingly capable AI systems.
Frontier models and the question of release A key issue is whether India should distinguish between ordinary AI applications and frontier models – systems at the cutting edge of capability that could pose greater risks if misused. Such a distinction could allow regulation to focus additional scrutiny on the most powerful systems rather than impose the same obligations on every AI developer. The US experience offers a possible reference point. The restrictions were subsequently lifted, with access beginning to be restored in July. One possible approach would be to require developers of particularly capable models to test them in a sandbox before wider deployment. Sandbox testing would allow researchers to assess a model’s behaviour and identify vulnerabilities before release. The precise nature of such restrictions, and the models to which they might apply, would need to be established through the legislative process. A central challenge will be defining what qualifies as a frontier model. From intermediary rules to stronger safeguards India has so far relied substantially on existing laws, rules governing online intermediaries and targeted interventions to address AI-related harms. The proposed law could go beyond regulating the content distributed through digital platforms to examine the systems that generate it. “The US has used executive powers in the absence of a legislation, the Indian system does not allow that. “If a company operates in India, it will have to follow Indian law. The framework could consider whether existing legal provisions need to be strengthened or whether specific obligations should apply to developers and deployers of advanced models Deepfakes, fraud and psychological harm The proposed framework is unlikely to be confined to frontier models. Deepfakes, financial fraud, impersonation, psychological harm and the growing difficulty of distinguishing authentic material from AI-generated content are also likely to feature prominently. Labelling may be one part of the response, but policymakers will also have to consider the context in which content is used and the harm it causes. A genuine photograph can be presented misleadingly, while an AI-generated image may be harmless. Human control and accountability Another likely focus is keeping humans in control of AI systems, particularly agents capable of carrying out tasks with limited supervision. Such systems can interact with software, access data and execute actions, creating risks that may not arise from a chatbot simply answering a question. The proposed framework could examine requirements for human approval of sensitive actions, restrictions on access to critical systems, monitoring, incident reporting and mechanisms to stop an agent when its behaviour becomes unsafe. Incidents involving Anthropic and OpenAI have also raised concerns about AI agents taking actions beyond the intended scope of their tasks. In cybersecurity evaluations, Anthropic’s Claude models accessed real third-party systems after testing environments mistakenly allowed internet access, while OpenAI disclosed that models in an internal evaluation exploited a vulnerability to gain internet access and reach systems at Hugging Face. In separate testing by the UK AI Security Institute, agents from both companies took unauthorised actions against real-world targets under conditions where internet access was enabled, and some safeguards were disabled. The incidents have highlighted the need for effective containment, monitoring and human oversight as AI systems become more autonomous. Innovation versus regulation The government is also considering how to avoid placing disproportionate compliance burdens on smaller developers and research institutions. A broad definition of frontier AI could inadvertently capture systems that do not present the same level of risk as the most advanced models. India’s approach may need to account for international practices while reflecting domestic legal requirements and the needs of Indian developers. “For India, the challenge is to develop safeguards that reflect domestic priorities while remaining compatible with international practices. The forthcoming consultation paper is expected to bring these issues into sharper focus.
Because computing requirements alone do not necessarily capture every risk, a threshold based on computing power used in training could offer one method, but capability-based assessments may also be needed. Because content moderation alone may not address risks such as autonomous AI agents taking unauthorised actions, models being used to identify vulnerabilities in critical infrastructure, or systems generating instructions that could facilitate harmful activity, this distinction matters. Restrictions imposed on Anthropic’s Mythos 5 and Fable 5 models in June, citing national security concerns, brought into focus the question of whether governments should intervene when AI systems acquire capabilities that could have national security implications. The Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021, provide a framework for platform obligations, while subsequent amendments and government directions have addressed synthetic content and deepfakes.
Officials familiar with the discussions said the government is examining how to address risks arising from increasingly capable AI systems, including models that can discover software vulnerabilities, enable sophisticated cyberattacks or generate convincing synthetic content. The definition of a “frontier model”, the safeguards expected of its developers and the scope of government oversight are likely to feature in discussions as the Centre prepares a consultation paper on AI regulation. Union Electronics and IT Minister Ashwini Vaishnaw has said that the government will release a consultation paper on a new AI law in the coming weeks. The paper is expected to invite views from civil society, citizens, industry and other stakeholders, he said. Officials familiar with the matter said pre-release evaluation, testing in controlled environments and safeguards proportionate to a model’s capabilities are among the issues likely to be discussed in India. “A vetting process before it rolls out” is one of the options likely to be discussed, officials familiar with the matter said. Officials familiar with the matter said the government could also examine whether developers should limit access to particular functions, restrict certain users or deploy different safeguards depending on the risks associated with a model. Officials familiar with the discussions said the relationship between existing intermediary rules and any new statutory safeguards for advanced AI is likely to be an important consideration. Officials said the government is thinking about “suitable enabling provisions” to address risks from advanced AI systems. There have to be legislative provisions for India to enact,” officials familiar with the matter said. The exercise behind releasing a consultation paper is to obtain consensus before introducing an overarching legislation,” officials said. Officials familiar with the discussions said “deepfakes”, “financial frauds” and “psychological harms” are among the areas that could be addressed in the consultation. Officials familiar with the matter said the government could examine whether platforms should be required to distinguish AI-generated material more clearly, how complaints and takedown mechanisms should operate, and how enforcement can account for the speed at which fabricated content spreads. Officials familiar with the discussions said the consultation could also examine the limitations of labelling alone and the risks posed by synthetic content presented as genuine. “Moreover, it’s not just about A.I. generated content, labelling could also be about letting users know they are accessing authentic material,” the official mentioned above added. Officials familiar with the discussions said the social implications of AI and the difficulty of exercising control over increasingly capable systems could form part of the regulatory debate. Officials familiar with the matter said the allocation of responsibility for harm caused by AI systems could also be considered, including the respective roles of model developers, companies deploying the technology, platforms providing access and organisations using its output. Officials familiar with the discussions said international coordination could also feature in the consultation, given that AI systems developed in one country can be deployed across jurisdictions. Excessively broad requirements could burden startups and researchers, while an overly narrow framework could leave serious risks unaddressed,” a second official said. Officials familiar with the matter said likely points of discussion include the definition of frontier models, pre-release assessments, sandbox testing, restrictions on high-risk capabilities, deepfake safeguards and the allocation of responsibility between developers, platforms and users.
