AI-generated pornography is proliferating rapidly, and the sharp rise of these deepfake images and videos has become an urgent concern.
Just about anyone can create deepfakes using a voice clip or partial images of a real person. The images, videos, or other content are digitally manipulated to make a convincing imitation of the victim.
A perpetrator generates misleading images and videos by using just one image or just one click, and anyone with a smartphone can create these images. They appear deceivingly real and are often complicated, if not impossible, to distinguish from reality. With technology being as advanced as today, it can be virtually impossible to discern between what’s a deepfake and what’s not.
From high-profile influencers and celebrities to small-town high school students or even young children, there are countless victims of AI-generated pornography. In fact, the FBI even warns that they’re overloaded with the recent rise in “sextortion” cases.
One recent incident involving high school students in New Jersey has brought attention to this increasingly problematic issue.
Fellow Student Makes AI Porn of New Jersey Teens
Francesca Mani, age 14, says a boy or group of boys at her school allegedly manipulated images of her and more than 30 other girls to create nonconsensual pornographic images. The deepfakes were reportedly shared in a group chat and then publicly online.
The images were created during the summer using an artificial intelligence app but were not discovered by Francesca until October. The Westfield High School student reported the incident to the principal. The boy who created the images was reportedly suspended for a few days and then returned to school.
The school believes the images have since been deleted and aren’t in circulation. The administration reportedly made counseling available for students and involved the Westfield police department in the ongoing investigation.
The principal also says they’re warning parents of the danger and prevalence of deepfakes among students. “New technologies have made it possible to falsify images, and students need to know the impact and damage those actions can cause to others.”
Francesca and her fellow students are calling for federal legislation to address and regulate the issue. They and US Intelligence officials argue that the gap between outdated legislation and our high-tech world needs to be bridged to better protect victims of AI-generated sexual images and prosecute offenders.
How common are deepfakes?
Individuals all over the world are consuming deepfakes at an alarming rate. In a recent study, independent researcher Genevieve Oh tracked the rise of deepfakes and found that 143,000 videos on 40 popular websites were viewed 4.2 billion times.
In 2019, researchers analyzed over 14,000 deepfake videos and found that 96% were pornographic. The report also showed that deepfake creation would double yearly.
AI-generated deepfakes use predominantly in porn
While deepfake images are AI-generated and used in various scenarios—like a seemingly harmless or funny Snapchat face swap—it’s most commonly used to exploit individuals via nonconsensual pornography.
The number of explicit deepfake videos on mainstream porn sites is rapidly increasing. And sadly, it seems as if some of the world’s most popular porn sites full of deepfakes refuse to take nonconsensual video creation seriously.
It can be virtually impossible to distinguish between real and AI-generated images. For many, seeing is believing, and even if the content they’re consuming features an individual not physically involved in the creation of the content, they’re exploited and traumatized just the same.
Victims of AI-generated porn are continuously violated every time someone clicks on or shares the nonconsensual images. For many, these images live forever and may continue to follow and haunt them throughout their life—like when applying for college or a job or when attempting to cultivate relationships. The images are even used as blackmail against them in some cases.
Deepfakes created without consent are a violation of digital privacy. And when the content created is pornographic, it can have an especially devastating impact on victims.
Are you or someone you know a victim of deepfakes? Check out these resources for things you can do to take action.
What can be done?
While it may seem insurmountable to help with an issue as rampant as deepfake AI-generated porn, there’s something significant you can do: help reduce the demand for nonconsensual pornography.
Think about it—the less people express a demand to consume pornography, the less need for deepfakes, and the more likely the supply for it will decrease over time.
No one deserves to be exploited and have images of themselves shared for sexual entertainment without their consent. You can help spread the word and stop the clicks.
Support this resource
Thanks for taking the time to read through this article! As a 501(c)(3) nonprofit, we're able to create resources like this through the support of people like you. Will you help to keep our educational resources free as we produce resources that raise awareness on the harms of porn and sexual exploitation?
DONATE