Indicting Deepfakes?: Why Gender-Based Violence Frameworks and Censorship aren’t the Only (or Best) Way to Respond to AI Generated Pornography

Recently, Twitch streamer Brandon “Atrioc” Ewing was caught accessing deepfake pornography of popular female Twitch streamers, including QTCinderella, Sweet Anita, Pokimane, and Maya Higa. For the uninitiated, deepfake refers to audio, image, or video files generated by artificially intelligent (AI) machine-learning applications (such as FakeAPP, DeepFaceLab, Zao, FaceApp, Wombo, DeepNude). Deepfake porn is created when AI is fed multiple image/video files of a person’s face which are then merged with images or videos of another person’s body to create original but fake products that appear authentic. Previously  limited to those with a high degree of tech savvy, the process is now accessible to anyone with a deep learning app and access to a cache of internet images.

After Ewing, apologized for paying to access deepfake porn via an ad he found on PornHub, some of the women whose images were deepfaked spoke out about their experiences. As with others whose images have been altered, they conceptualized their experiences as uninvited objectification and sexual violation. QTCinderalla released a tearful livestream response exclaiming that “this is what pain looks like…This is what it looks like to feel violated. This is what it feels like to be taken advantage of, this is what it looks like to see yourself naked against your will being spread all over the internet.” Sweet Anita is quoted as claiming, “There isn’t any moving on from it. This is just out there forever now. This is just an extra aspect that I will have to deal with, and there’s no taking it back.”

The Twitch deepfake controversy has refuelled calls for the greater regulation, censorship, and the criminalization of deepfake porn. For instance, following the outrage provoked by a deepfake of Gal Gadot in 2017, Reddit updated its community guideline’s ban on the sharing of pornographic material without the permission of the depicted person­­ to include “depictions that have been faked”. In 2018 Google followed suit, adding “involuntary synthetic pornographic imagery” to its list of banned items and committing to blocking search results that falsely depict individuals as “nude or in a sexually explicit situation” upon request. In August of 2020, TikTok updated its policy, prohibiting synthetic or manipulated content which “misleads users by distorting the truth of events and cause harm to the subject of the video, other persons, or society.”

As noted above, these regulatory responses are motivated, in part, by individuals’, scholars’, and lawyers’ classification of deepfakes as a form of gender-based violence. In a recent report on Deepfakes by the Amsterdam-based cybersecurity company Deeptrace, Danielle Citron, one of the most oft cited legal commentators on this topic states: “Deepfake technology is being weaponized against women by inserting their faces into porn. It is terrifying, embarrassing, demeaning, and silencing. Deepfake sex videos say to individuals that their bodies are not their own and can make it difficult to stay online, get or keep a job, and feel safe.” Since then, four states in the US have passed deepfake laws, the UK is planning to criminalize deepfakes, and Korea has already declared them a sex crime.

But is gender-based sexual violence the best way to make sense of deepfake porn? Is censorship and criminalization the best possible response? Without denying that deepfakes may pose a potential threat to commerce and democracy and that deepfake pornography may be subjectively experienced as emotionally distressing, a violation of one’s sexual autonomy, or as a potential threat to one’s economic livelihood, there exists other, equally legitimate, ways of making sense of deepfake porn, including conceptualizing it as manifest sexual fantasy, as remix culture, and as celebrity fan art, only the first of which is briefly considered below.

Deepfake Porn as Sexual Fantasy

Sexual fantasy is an integral part of many peoples’ sex lives yet it is rarely acknowledged, let alone lauded, in debates about synthetic media and the production of sexually explicit content.

A recent large mixed method Canadian study about the nature and intensity of “men and women’s” sexual fantasies found that all genders fantasize regularly and that men tend to report a higher diversity of sexual fantasies than women.Having sex with someone other than the respondent’s current partner was rated significantly higher for men than for women, and both women and men commonly report fantasies that specifically refer to an authority figure or a celebrity.

To date we know very little about deepfake porn creators or consumers yet they are widely constructed as depraved, as the embodiment of “toxic geek masculinity, and as lacking empathy. What we know from clinical practice suggests that misogyny rarely serves as a key motivator in those who create and encounter various kinds of “fake porn.” As clinical psychologist Dr David J. Ley argues, the overwhelmingly negative media attention given to deepfake porn, “creates a false, two-dimensional image of a complex phenomenon.” According to Ley, “far more of these cases [are] driven by feelings of loss, shame, hope, and fantasy than by misogyny and anger.” Rarely, however, are creators of deepfake pornography compared to the cut and paste or photoshopped “porno collages” of yesteryear—where the faces of female crushes were pasted onto pornographic images as a means to “explore fantasies that were likely impossible for them to ever fulfill”. And, while for some, maintaining these fantasies inside their own heads or homes serves them well, Ley notes that others want to share their fantasy sexual creations with others for a variety of reasons, from seeking approval, to demonstrating how they’ve overcome technical challenges, to playing around with taboo, to bonding and connecting with those who share the same interest or arousal triggers, and/or to arouse others in the way they are aroused so as to feel less alone for having these interests and desires.

How to make sense of deepfake porn is obviously more complex than can be captured in a short blog post. Sexual fantasy research is similarly complex yet nevertheless presents an opportunity to reframe, de-exceptionalize, and destigmatize deepfake porn. The construction of deepfake porn and/as sexual fantasy overlaps with long seated debates about the censorship and criminalization of pornography and/as sexual violence, but failing to acknowledge the salience of sexual fantasy and the limits of gendered-based sexual violence frameworks for making sense of this issue comes with its own harms—including the marginalization of non-normative sexual communities and practices, the surveillance and criminalization of racialized and marginalized men, and the ever expanding concept of sex crime and criminal law.

Returning briefly to the issue of sexual fantasy, it’s worth noting that exploring one’s sexual fantasies via the use of the Internet and new sexual technologies is increasingly being conflated with, and criminalized as, abuse or as inchoate crime. (Gilden, 2016). Given that, according to Gilden, “tension between protected sexual identity and marginalized sexual fantasy has become particularly acute” in the legal realm, and that “judges and juries in several areas of the law repeatedly conflate sexual fantasy with sexual abuse, have largely been dismissive of both the merits and value of fantasy based defenses, and have relaxed evidentiary standards in ways that particularly prejudice individuals whose desires likely provoke disapproval or disgust”, there exists reason for sextech producers and consumers of all ilk to be concerned.

Consent, freedom of expression, freedom from sexual violence, public sex, public status, and sexual fantasy are all factors that communities and courts will have to grapple with when balancing the personality rights of one individual with the freedom of expression rights of another. Undoubtedly, concerns about autonomy and self-determination remain key. Nevertheless, alternative frameworks for making sense of and responding to deepfake porn should be considered before we move to censoring and criminalizing deepfake producers, consumers, and products, particularly given the sex-negative, pornphobic, and whorephobic context within which we currently reside.