San Francisco City Attorney Mandates Removal of "Nudify" Apps from Apple and Google Stores, Citing Illegality and Exploitation

San Francisco City Attorney David Chiu has issued a stern directive to tech giants Apple and Google, demanding the immediate removal of dozens of "nudify" applications from their respective App Store and Google Play platforms. This decisive action comes with a significant warning: both companies could face severe civil penalties for allegedly violating California law by facilitating the distribution and monetization of apps that generate non-consensual deepfake pornography. The instruction, conveyed through formal letters reviewed by TechCrunch, underscores a growing legal and ethical battle against the misuse of artificial intelligence for harmful purposes, particularly involving the exploitation of women and girls.

These problematic applications leverage sophisticated AI algorithms to digitally alter photographs, transforming images of clothed individuals into depictions of nudity without their consent. Crucially, these apps are not only available for download through Apple and Google’s ubiquitous app stores but also process payments via the platforms’ integrated billing systems, effectively making Apple and Google direct beneficiaries of the transactions. The letters from Chiu’s office grant the companies a 28-day window to respond and demonstrate compliance. In initial reactions, Apple confirmed the removal of three specified apps and initiated the termination of associated developer accounts, while Google reported the suspension of all five referenced apps on the Play Store.

The Alarming Rise of AI-Generated Non-Consensual Imagery

The proliferation of "nudify" apps represents a disturbing facet of the broader challenge posed by generative artificial intelligence. While AI holds immense potential for innovation, its misuse in creating deepfake imagery has become a significant societal concern. Deepfakes are synthetic media in which a person in an existing image or video is replaced with someone else’s likeness. When applied to create non-consensual intimate imagery (NCII), the impact on victims is profound and devastating, ranging from severe psychological trauma and reputational damage to social ostracization and loss of control over one’s digital identity.

The technology behind these "nudify" apps has become increasingly accessible and user-friendly, allowing individuals with minimal technical expertise to generate highly convincing, yet entirely fabricated, nude images from ordinary photographs. This ease of access, combined with the anonymity often afforded by digital platforms, has fueled a cottage industry of exploitation. Statistics on the prevalence of deepfakes paint a grim picture; a report by Sensity AI in 2023 indicated a significant year-over-year increase in deepfake content, with non-consensual intimate deepfakes constituting a substantial portion of this growth. The vast majority of victims of such content are women, highlighting a gendered dimension to this form of digital abuse.

App stores, by virtue of their near-monopoly on mobile software distribution, serve as critical gatekeepers. Their policies typically prohibit content that is sexually explicit, harassing, or exploitative. However, the sheer volume of applications and the evolving nature of AI technology present formidable challenges for effective content moderation. Furthermore, the revenue-sharing model, where app stores take a percentage of in-app purchases and subscriptions, places them in a position where they directly profit from every transaction, including those for illicit services. This financial incentive, as highlighted by City Attorney Chiu, forms a central pillar of the legal argument against Apple and Google.

A Chronology of Warnings and Legislative Action

The current enforcement action by the San Francisco City Attorney’s office is not an isolated event but rather the culmination of sustained pressure and prior warnings from watchdog organizations. A clear timeline of events underscores the platforms’ alleged awareness of the issue:

• January 2026: The Tech Transparency Project (TTP), a non-profit watchdog group, published its first comprehensive report detailing the widespread availability of "nudify" apps on both Apple’s App Store and Google Play. This initial report identified dozens of applications that offered non-consensual intimate image generation services, often for a fee processed directly by the app stores. Following the publication, TTP formally sent letters to both Apple and Google, explicitly notifying them of their findings and the presence of these problematic apps.
• April 2026: Undeterred by what it perceived as insufficient action, the TTP released a second, follow-up report. This subsequent investigation revealed that many of the previously identified "nudify" apps remained available, and new ones had emerged. The April report intensified its criticism, asserting that Google and Apple were not only passively hosting these apps but, in some cases, actively guiding users toward them through search algorithms and recommendations. It characterized both companies as "key players in the spread of AI tools that can turn real people into sexualized images." TTP again dispatched letters to Apple and Google, reiterating its concerns and providing further evidence.
• Present Day (Late 2026): Leveraging the prior notifications from TTP, San Francisco City Attorney David Chiu initiated his official legal intervention. His office sent formal letters to Apple and Google, citing the TTP reports as evidence of the companies’ long-standing awareness of the issue. The letters serve as a direct warning of potential civil penalties under California law, giving the tech giants a strict 28-day deadline for a comprehensive response and demonstrable action.

This timeline highlights a critical aspect of Chiu’s legal strategy: establishing that Apple and Google were not merely unaware hosts but had received explicit, repeated warnings about the illegal activities transpiring on their platforms for nearly a year. This prior knowledge is crucial in establishing liability under California statutes.

The Legal Framework: California’s Stance Against Deepfake Exploitation

California has been at the forefront of legislative efforts to combat the harms of deepfake technology, particularly in the realm of non-consensual intimate imagery. Chiu’s letters to Apple and Google reference two key statutes, one existing and one soon to take effect, that form the legal bedrock of his demand:

1. Existing Law (Crime to Facilitate Non-Consensual Deepfake Pornography): California law currently makes it a criminal offense to "knowingly facilitate or recklessly aid" in the creation or dissemination of non-consensual deepfake pornography. The terms "knowingly facilitate" and "recklessly aid" are central to the City Attorney’s argument against Apple and Google. By hosting these apps, processing payments, and allegedly failing to act despite repeated warnings, Chiu’s office contends that the companies have, at minimum, recklessly aided in the criminal activity. Their platforms act as conduits, and their revenue streams from these apps could be interpreted as a form of facilitation. The legal standard of "recklessly aid" does not necessarily require intent to cause harm but rather a disregard for a substantial and unjustifiable risk of harm.
2. Upcoming Law (2025 Civil Action for Victims): A separate, powerful law is set to take effect in California in 2025. This statute will empower victims of non-consensual deepfake pornography to pursue civil action against "third-party facilitators" of such content. This is a groundbreaking development, as it provides a direct legal avenue for victims to seek damages and redress from platforms or individuals who enable the creation and spread of these harmful images. By citing this upcoming law, Chiu’s office sends a clear signal about the increasing legal accountability for platforms and the expanding rights of victims. It serves as a pre-emptive warning that continued inaction could lead to a cascade of civil lawsuits once the law is active.

City Attorney Chiu explicitly articulated the gravity of the situation in an email to TechCrunch: "Apple and Google are profiting from apps that exploit women and girls by creating non-consensual intimate deepfakes." He further emphasized, "While the companies have removed some problematic apps, they still have a responsibility to be proactive and vigilant in preventing sexual abuse." The letters also highlight that both companies had been aware of their role in processing payments for these apps for nearly a year, an assertion corroborated by the TTP reports. Chiu told Wired that the companies had "likely earned millions of dollars in fees" from these "nudify" services, underscoring the financial dimension of their alleged complicity.

Official Responses and Lingering Questions

In the immediate aftermath of Chiu’s directive, both Apple and Google issued statements regarding their actions:

• Apple’s Response: An Apple spokesperson informed TechCrunch that "nudify apps are not allowed on the App Store under existing policies." The company confirmed, "We have removed three of the apps in question and are in the process of terminating their developer accounts." Furthermore, Apple stated it was "in contact with four others that need to fix policy violations or they may be removed." While these actions indicate compliance with specific demands, Apple did not provide details on the total revenue processed through these illicit apps or the aggregate number of downloads they garnered prior to removal. This lack of transparency leaves open questions about the true scale of the problem on their platform.
• Google’s Response: A Google spokesperson confirmed that all five Play Store apps specifically mentioned in Chiu’s letter had been suspended. "When violations are reported to us, we investigate and take swift action," the spokesperson stated. "In the case of these apps, that has included suspending hundreds of violating apps and restricting related search terms like ‘nudify’ on our store." Google’s claim of suspending "hundreds of violating apps" suggests a broader, albeit perhaps reactive, effort beyond just those named by Chiu. However, similar to Apple, Google refrained from disclosing any figures related to total revenue or downloads from these apps.

The focus of this crackdown appears to be primarily on the store-level distribution and payment processing mechanisms. It addresses the direct availability of these apps within the established ecosystems of Apple and Google. However, it does not directly tackle the underlying AI models or websites that host similar tools outside the confines of the app stores. This distinction is crucial, as the core technology for generating deepfakes often exists independently and can be accessed through other means, posing a continuous challenge to content moderation efforts that extend beyond app store policies.

For users who purchased or subscribed to any of the affected "nudify" apps, the termination of developer accounts may lead to a loss of access. Neither company has yet shared details regarding potential refunds for these users, which could become another point of contention. Developers who create apps combining photo editing with generative AI features for iOS or Android are now under increased scrutiny. They are strongly advised to meticulously review the specific policies cited by Apple and Google. Apple’s App Store guidelines already contain restrictions on content that sexualizes real people without consent, while Google has proactively restricted search terms like "nudify" across the Play Store. Apps that generate edited images of identifiable individuals without robust safeguards against non-consensual use are at a significantly higher risk of removal from both platforms.

Broader Implications and the Future of Platform Accountability

San Francisco City Attorney David Chiu’s directive marks a significant escalation in the ongoing struggle for platform accountability in the digital age. This action sets a powerful precedent for holding major tech companies responsible for the content distributed and monetized through their ecosystems, especially when state laws are violated. It underscores a growing judicial and public expectation that platforms must move beyond reactive content moderation to a more proactive and vigilant stance against harmful and illegal activities.

The implications extend far beyond "nudify" apps. This case could influence how platforms approach content moderation for a wide array of problematic AI-generated content, from misinformation and hate speech to other forms of exploitation. It highlights the inherent challenges of moderating at scale, particularly with rapidly evolving AI technologies that can generate content indistinguishable from reality. The need for sophisticated AI detection tools and robust human oversight will only grow.

Legislatively, California’s proactive stance, especially with the 2025 law, signals a broader trend. Other states and potentially the federal government may look to California’s model in crafting their own legislation to address the ethical and legal quagmires presented by generative AI and deepfakes. This could lead to a more fragmented, yet ultimately more stringent, regulatory landscape for tech companies across the United States. The pressure on AI developers will also intensify, urging them to embed ethical considerations and safeguards against misuse directly into their models from the earliest stages of development.

Ultimately, this move by the San Francisco City Attorney is a critical step in strengthening victim support and rights in the digital realm. While legal avenues for redress are increasing, the emotional and psychological toll on victims of non-consensual deepfake imagery remains severe. The ongoing battle against the misuse of AI will require a concerted effort from lawmakers, tech companies, advocacy groups, and the public to ensure that technological advancement does not come at the cost of human dignity and safety.

How to Report Nudify Apps and Protect Yourself

For users who encounter or identify apps of this type on either the Apple App Store or Google Play, reporting them directly is a crucial step in combating their proliferation. Both platforms provide dedicated tools for users to flag inappropriate content:

• On iOS (Apple App Store): Users can find the "Report a Problem" option directly on each app listing page. This feature allows users to categorize the nature of the problem, and when reporting non-consensual intimate imagery generation, it is important to reference that specific policy category.
• On Android (Google Play Store): The "Flag as inappropriate" option is located within the app listing, typically under the developer contact section. Similar to iOS, selecting the most relevant policy violation category is essential for effective reporting.

Victims of non-consensual deepfake imagery in California have an increasingly robust legal framework to protect them. As mentioned in Chiu’s letters, the law set to take effect in 2025 will provide the option to pursue civil action against third-party facilitators. This legal avenue offers a pathway for victims to seek justice and compensation for the harm they have suffered.

The 28-day deadline for Apple and Google to respond to the City Attorney’s letters is fast approaching. So far, neither company has publicly confirmed whether they plan to conduct a broader, proactive review of related apps beyond those specifically named by Chiu or the Tech Transparency Project. This proactive step would be vital in demonstrating a genuine commitment to preventing sexual abuse and upholding their platform policies. The outcome of this legal challenge will undoubtedly shape the future of content moderation and platform accountability in the age of artificial intelligence.