Centre planning new regulations, penalties for both creators and platforms to deal with deepfakes

The minister, who met social media platforms on the deepfake issue on Thursday, said that companies have agreed on the need for clear actionable work in areas such as detection, prevention, strengthening of reporting mechanism, and raising user awareness.

New Delhi: The government is planning new regulations that may impose penalties on both creator and platform hosting deepfakes as it looks to clamp down on what IT and Telecom Minister Ashwini Vaishaw described as "a threat to democracy".

Amid some celebrities reporting their faces being manipulated onto another video, new protection regulations being considered will look at measures including watermarking AI-generated content, deepfake detection, rules for data bias, privacy and guards against concentration.

"Deepfakes have emerged as a new threat to democracy. These (can) weaken trust in society and its institutions," Vaishnaw said after meeting with various stakeholders, including social media platforms, Nasscom and other professors from the field of artificial intelligence (AI).

"We will start drafting the regulations today itself and within a very short timeframe, we will have a separate regulation for deepfakes," he said.

The government, he said, would come up with actionable items on four pillars -- detection of deepfakes, preventing the spread of such content, strengthening reporting mechanisms, and spreading awareness on the issue -- within 10 days.

All stakeholders present at the meeting shared similar concerns regarding deepfakes, he said. "All social media platforms agreed to have extensive technology to detect deepfakes."

India has over 80 crore internet users, which are projected to cross 120 crore in two years. Deepfake is a piece of technology that leverages AI to alter a person's appearance, voice, or actions in a way that can be realistic and challenging to discern from authentic, unaltered content. Recent deepfakes have brought to the fore the urgency of a regulatory framework for AI in the new Digital India law.

Vaishnaw said deepfake advertisements or misleading promotions are a threat that Indian society is facing currently.

"The use of social media ensures that deepfakes can spread rapidly in a more significant manner without any check and go viral. This is why we need to take urgent steps to strengthen trust in society and our democracy," he told reporters here.

Deepfakes shot into prominence after actor Rashmika Mandanna's face was found to have been used in an embarrassing video earlier this month. Some other celebrities including Katrina Kaif and Kajol were also reported to be victims of deepfake.

Last week, Prime Minister Narendra Modi also warned about the threat deepfakes pose.

On Saturday, Vaishnaw warned social media platforms would lose the immunity they enjoy under the 'safe harbour' clause in the Information and Technology Act if they fail to take measures against deepfakes. The clause states that an online platform cannot be held accountable for the content shared on it by users.

After the meeting with stakeholders on Thursday, he said deepfake video creators have found ways to even crack labelling and watermarks. "Thus, there has to be something that finds a way out of it."

Next meeting on the subject will be held in the first week of December.

Within the next 10 days, the government would come up with clear actionable items on four pillars -- detection (of deepfakes, misinformation), how to prevent spread of misinformation, how to strengthen reporting mechanisms, (in-app reporting mechanisms have to be strengthened) and increasing awareness, the minister said.

"All the companies have shared our concern. They understood that this is not free speech, that this is something very harmful... they have understood the need for much heavier regulation," he added. "The use of social media is ensuring that deepfakes can spread significantly more rapidly without any check and they are getting viral in few minutes of the uploading.

The minister said there are very urgent steps need to be taken to strengthen trust in society and to protect our democracy. "There is a need to take steps on this at the earliest whether they are legal, regulatory or technical action we need to take all sorts of steps.

Asked if there will be a change in the existing rule or new law may be brought, he said, "We can bring this in the form of making amendments to the existing rules or we can bring a new set of regulations.

"We also discussed watermarking and labelling. All agreed that we have to do this, this is the basic minimum which all will have to do," he said.

"When we will draft the regulation we will also be looking at the penalties both the person who has uploaded or created as well the platform. We are saying that the government will bring the regulation for detection, prevention, strengthening the reporting mechanism and creating awareness and using technologies for deepfakes and AI-generated content which can be harmful to society.

Until the regulation is made, social media platforms and companies promised to take all possible measures to prevent the spread of deepfakes. "All of them said they are taking steps internally and they would like to increase the intensity of those steps," he said.

Stating that free speech and privacy are both important for the government, he said both these constructs are being undermined with deepfake. "So new regulation is for deep fake and AI-generated content is not harmful to society."

Giving examples of deepfakes, the minister said during electioneering in Madhya Pradesh, a video surfaced in which the chief minister was kind of saying to vote for the opposite party.

"That was absolute misinformation, deep fake and deep misinformation. We have to address that apart we have to ensure that the people who create these they are identified they have their own set of punishments simultaneously the platforms which are the tools through which this content is spreading they also have to take responsibility in terms of what they are allowing to be out on their platforms.

"Detecting a deepfake is very important. It is very important to identify between synthetic and deepfake content," he added.