Meta’s Oversight Board, a semi-independent policy council, has recently focused on how the company’s social media platforms manage explicit, AI-generated images. On Tuesday, the Board announced investigations into two distinct cases involving Instagram in India and Facebook in the U.S., where Meta’s systems failed to effectively detect and address explicit content generated by AI.
The Cases Under Review
The first case involves an AI-generated nude image of a public figure from India, which was reported as pornography on Instagram. The image was posted by an account specializing in AI-generated images of Indian women, predominantly viewed by users in India. Meta initially failed to remove the image following the first report, and the report was automatically closed after 48 hours without further review. Even after the complainant appealed, the image remained on the platform until the Board intervened. Meta eventually removed the image for violating its community standards on bullying and harassment.
The second case pertains to Facebook, where a user posted an explicit AI-generated image resembling a U.S. public figure in an AI-focused group. In this instance, Meta had already taken down the image and categorized it under “derogatory sexualized photoshop or drawings” in its Media Matching Service Bank.
The Oversight Board chose these cases not necessarily because they were the most egregious but because they represent broader issues across Meta’s platforms. By examining cases from different regions, such as the U.S. and India, the Board aims to evaluate the global effectiveness of Meta’s policies and processes.
Addressing Deepfake Porn and Gender-Based Violence
Generative AI tools have recently allowed users to create explicit content, including deepfake pornography. Reports indicate that such content has become a significant issue, particularly in countries like India. Last year, the BBC reported a surge in deepfake videos featuring Indian actresses. Indian Deputy IT Minister Rajeev Chandrasekhar expressed dissatisfaction with tech companies’ responses to deepfakes, warning that the government could block platforms that fail to address these issues adequately.
While India has provisions for reporting online gender-based violence, experts argue that the process is cumbersome and lacks adequate support. Advocacy groups like IT for Change highlight the need for more robust legal processes to handle online gender-based violence effectively.
Experts such as Aparajita Bharti from The Quantum Hub suggest that AI models should be restricted from generating explicit content that could cause harm. She advocates for training AI systems to prevent harmful content creation and implementing default labeling to enhance detection.
Meta’s Response and Future Steps
In response to the Oversight Board’s findings, Meta removed the problematic content from both Instagram and Facebook. However, the company has not addressed the delay in content removal or the effectiveness of its initial moderation efforts. Meta claims to use a combination of AI and human review to manage sexually suggestive content and ensures such content is not promoted in areas like Instagram Explore or Reels recommendations.
The Oversight Board has invited public comments by April 30 to address the harms of deepfake pornography and evaluate Meta’s approach to detecting AI-generated explicit content. The Board will review these comments and update its findings in the coming weeks.
These cases underscore the ongoing challenge for large platforms in managing user-generated content, particularly as AI technologies enable new forms of content creation. While Meta is working on AI tools to identify and manage explicit content, the effectiveness of these systems remains a critical concern. Platform policy experts point out that labeling, while useful, often falls short in curbing the spread of harmful content.
