Apple Tightens App Store Rules for AI Image-Manipulation Apps

Apple is extending its App Store rules beyond apps built explicitly to create sexual images. Developers of apps that could generate nonconsensual intimate imagery of real people must now take steps to prevent that material from being uploaded or shared, according to a note on Apple’s developer website. The change shifts more responsibility for detection and cleanup onto app makers.

That distinction matters because harmful image-generation features can sit inside products presented as general-purpose editing or user-content tools. Apple already bars pornographic material and apps designed specifically to produce it. The new requirements address apps whose broader functions could still be used to create intimate images without a person’s consent.

What developers must do

Under the updated requirements, affected apps must filter this content, offer users a way to report it, prevent uploads and remove material that slips through those protections. These are separate steps in a moderation process: filtering is meant to catch prohibited material, upload controls aim to stop it from entering the service, and reporting gives users a route to flag failures. Removal is the backstop when prevention doesn’t work.

For developers, the practical change is not simply to add a warning or revise an app description. They need measures that operate within the product, including a reporting path and a process for responding when content gets past the initial checks. Apple’s note doesn’t specify a particular technical approach, so developers will need to determine how to meet the requirements in the context of their own features and services.

That leaves room for different implementations, but it also raises questions teams will have to answer themselves: what content their filters can identify, how users can report it, and how quickly flagged material can be reviewed and removed. The source doesn’t set out technical thresholds or a review timetable. Those details may matter especially for apps that allow users to upload images and then apply AI-based transformations.

App review now carries more risk

Apple says apps that fail to follow the rules can be removed from the App Store. Developers may also lose the ability to submit apps. That makes compliance a continuing product responsibility, rather than a one-time exercise during initial review. A feature that appears acceptable in a submission could still create problems if the app’s actual capabilities or moderation practices don’t match what Apple expects.

The company has also added a rule against “bait and switch” tactics. In this context, that means misrepresenting an app’s capabilities during the submission review period. The provision gives Apple another basis to act when an app’s review presentation doesn’t reflect what users can do with it. For teams, the straightforward implication is that descriptions and submitted builds need to accurately represent the product, including features that could enable image manipulation.

Apple’s announcement describes the requirements, but doesn’t explain how it will assess each app’s safeguards or how developers should handle borderline cases. Teams building general image tools will therefore need to examine not only the intended use of their features, but also plausible misuse. A moderation system that works for ordinary uploads may not be enough if the product can generate or alter intimate imagery.

A response to apps that slipped through

The change follows a study earlier this year that found dozens of apps capable of creating deepfake nude images of real people on both Apple’s App Store and Google Play. Apple removed many of the apps identified at the time. The new requirements suggest that removing individual listings hasn’t settled the issue: similar capabilities can appear again, including in apps that aren’t described as dedicated image-generation tools.

That history also points to the limits of relying on app review alone. Review can assess a product at submission, but user-generated content and changing app features create ongoing moderation needs. Apple is now making developers responsible for more of that work inside their products, while retaining the option to remove apps and restrict future submissions.

For product teams, the immediate task is to map potentially risky image features to concrete controls: prevention, filtering, reporting and removal. Apple hasn’t published implementation specifications in the note, so the next test will be how consistently it applies the rules and what level of evidence it expects from developers. The practical takeaway is clear: apps with image tools need safeguards that work after launch, not just a careful description at review.