San Francisco Orders Apple and Google to Remove Deepfake Nudity Apps, Citing Illegality and Profiteering.

San Francisco City Attorney David Chiu has issued a stern directive to tech giants Apple and Google, demanding the immediate removal of applications from their respective app stores that enable users to create non-consensual deepfake nude images. The cease-and-desist letters, reviewed by WIRED, explicitly instruct the companies to cease profiting from this harmful technology, which facilitates the "nudification" or "undressing" of individuals through AI.

On Thursday, Chiu’s office dispatched legal notices to both Apple and Google, specifically targeting 13 face-swapping apps. Eight of these applications were found on Apple’s App Store, while the remaining five were available on Google’s Play Store. These apps, the letters contend, allow users to generate AI-powered intimate images without the consent of the depicted individuals. The legal correspondence further accuses the Silicon Valley behemoths of "aiding and abetting" the distribution and sale of explicit deepfake content, urging them to "sever" all business relationships with the developers of these illicit applications.

"Generating non-consensual intimate images is illegal, harmful, and completely unacceptable," Chiu stated in an interview with WIRED. He underscored the severe ethical and legal ramifications of these applications, highlighting that his office had previously taken legal action against 16 prominent deepfake websites. Chiu also estimated that Apple and Google have likely "made millions of dollars in fees" from these "nudification" apps through their in-app payment systems, from which the tech companies typically take a significant cut. He emphasized the urgent need for both companies to enhance their content moderation processes to prevent such harmful applications from appearing in their stores in the first place.

Chiu stressed the inherent responsibility of these platforms, asserting, "These companies have responsibility to ensure that apps on their platforms do not facilitate sexual abuse." The legal letters from the city attorney’s office explicitly reference California law, specifically California Civil Code section 1708.86, which prohibits supporting services that contribute to the creation and dissemination of deepfake pornography. The letters underscore the gravity of the situation, stating, "The fact that some of the world’s largest and most established technology companies are facilitating this has to stop."

The pervasive issue of "nudify" apps is not a new discovery for Apple and Google. Independent researchers and watchdog organizations have repeatedly identified and reported the widespread availability of such applications across both the App Store and Google Play. These reports consistently demonstrate that apps capable of generating sexual images using artificial intelligence are easily accessible, with some even being mistakenly rated as "suitable for use by children," thereby exposing younger users to potentially harmful content. Despite the emergence of new laws and bans designed to combat the proliferation of explicit deepfakes online, technology and social media companies frequently direct millions of users toward this dangerous technology, often inadvertently.

Both Apple and Google maintain comprehensive developer policies that explicitly prohibit pornography, abuse, and harassment on their respective platforms. In the past, following reports from researchers and journalists, both companies have taken action to remove dozens of "nudify" and deepfake apps. However, the persistence of these applications indicates a significant challenge in enforcement and proactive identification.

Responding to the allegations, Google spokesperson Dan Jackson informed WIRED that the company has already deleted "hundreds" of apps featuring "nudifying" capabilities due to policy violations. This includes the five Android apps specifically flagged by Chiu’s office, alongside other measures implemented to restrict access to them. "Google Play does not allow apps that contain sexual content, and we continually take proactive steps to detect and remove apps with harmful content," Jackson affirmed in a statement. He added, "When violations are reported to us, we investigate and take swift action, which in the case of these apps has included suspending hundreds of violating apps and restricting related search terms like ‘nudify’ on our store." Apple, however, did not provide a comment prior to the publication of the report.

The last five years have witnessed an alarming surge in highly lucrative deepfake "nudification" technology online. This phenomenon gained significant mainstream attention in January when xAI’s Grok AI chatbot was reportedly used to generate millions of sexualized images. A multitude of apps, websites, and bots now allow individuals—predominantly men—to upload images of others—overwhelmingly women and girls—and digitally "remove" their clothing or even place them into graphic sexual scenarios. The ease with which these deepfakes can be created is particularly concerning; often, all that is required is a reference photo and a few clicks, with results frequently available within seconds. The realism of these images and videos has dramatically improved as the underlying generative AI technology has advanced, with many services offering some results for free or charging minimal fees for the creation of this harmful content. Previous investigative reporting by WIRED and Indicator Media has tragically uncovered incidents in at least 90 schools where deepfake sexual abuse images of minors have been created and circulated, highlighting the profound and widespread impact of this technology.

Chiu passionately articulated the devastating human cost of this technology, stating, "These images are used to bully, humiliate, and threaten women and girls." He further elaborated on the profound and lasting damage inflicted upon victims: "This industry has a horrific impact on one’s reputation, mental health, loss of autonomy. There have been victims who’ve been suicidal."

The 13 apps that were the focus of the City Attorney’s Office investigation—eight on Apple’s App Store and five on Google’s Play Store—typically market themselves broadly as "face-swapping" tools. However, their deeply concerning capability to create sexual deepfakes becomes apparent and accessible once users engage with them. For instance, the website of one app, which boasts over 1 million downloads, prominently displays more than a dozen distinct styles of AI images it can generate. These include overtly sexualized descriptions such as "bikini queen curvy," "calm busty," and "cinematic intimacy," often accompanied by illustrative images of sexualized women. The homepage of another targeted app brazenly advertises its ability to produce "free and uncensored" videos. WIRED has refrained from naming these specific applications to avoid inadvertently promoting them.

The continued presence of these problematic apps on their platforms is unlikely to surprise Apple and Google. Over the past year, multiple independent reports have consistently identified applications in both companies’ app stores that facilitate the creation of nonconsensual nude images or videos. In January and April of this year, the Tech Transparency Project (TTP), a respected independent watchdog group, uncovered approximately 100 such apps across both the App Store and Play Store, along with advertisements for "nudifying" technology appearing on the platforms themselves. Google’s spokesperson Jackson stated that the company has since removed the majority of the apps identified by TTP. The apps highlighted by TTP’s research were collectively estimated to have amassed around 480 million downloads and potentially generated approximately $120 million in combined revenues, underscoring the significant financial incentive behind their development and distribution.

Katie Paul, the director of TTP, expressed her dismay at the persistent problem. "We didn’t think after the first report that we would see this as a problem again—and it was just as bad, if not worse, after the second report," she remarked. Paul criticized the discrepancy between the tech giants’ marketing and reality: "Apple and Google make a lot of promises in their marketing about how trusted and safe their app stores are. And that is just not what is playing out in reality."

Adding to the body of evidence, a preprint research paper published in May by researchers from Cornell University and Georgetown University identified a staggering 420 apps offering general face-swapping functionalities across both Google’s and Apple’s app stores. Out of these, 155 were rigorously tested to ascertain their capacity for creating face swaps with nude images. The findings were alarming: in 70 percent of the tested cases, it was possible to create such content, with the apps notably lacking any effective safety measures to prevent this misuse. "None of these apps are advertised as nudification apps," the research paper explicitly states. "This suggests that face swap apps, and many other forms of AI image generation and editing apps, are effectively ‘dual-use’: apps that evade content moderation by platforms because they present as benign, but possess the capability to create harmful content."

San Francisco City Attorney David Chiu reiterated his office’s unwavering commitment to addressing this pressing issue. He described being "absolutely horrified" by the profound harm and vast scale of this technology. "My hope is that Apple and Google will immediately remove these apps and strengthen their screening systems to make sure that apps like this never get onto their platforms in the future," Chiu stated, outlining his immediate expectations. He concluded with a clear warning regarding potential future actions: "It’s our hope that these companies will do the right thing—but if they don’t, we will have to consider all of our legal options."