Alarming rise in AI-generated child abuse material, regulator warns

The negative consequences of artificial intelligence, which have already led to the spread of fake and deepfake celebrity videos online, are now extending to children. AI-generated child sexual abuse content is increasingly found on publicly accessible parts of the internet.

According to a report by the Internet Watch Foundation (IWF), AI-generated child sexual abuse content is exposing even more people to the harmful and gruesome images.

Many of the images and videos of children being injured and abused are so realistic that they are difficult to distinguish from images of real children. They are considered criminal content in the eyes of UK law, just as ‘traditional’ ‘child sexual abuse material would be according to the IWF.

In the past six months alone, IWF analysts have seen a 6% increase in confirmed reports of AI-generated child sexual abuse material, compared to the previous twelve months.

The IWF warns that almost all of the content (99%) was found on publicly available parts of the internet and was not hidden on the dark web. Most reports come from members of the public (78%) who have come across the criminal images on sites such as forums or AI galleries. Otherwise, action was taken by IWF analysts through proactive search.

Analysts say that viewing AI-generated content of sexually abused children could be as disturbing as seeing real children in abuse images if someone is not prepared or trained to handle seeing such material.

Certain AI child sexual abuse material is classified as non-photographic images, such as cartoons, and is also considered harmful to view and rated accordingly by IWF analysts.