Blog:
Evaluating Google Cloud Vision for Image Moderation: How Reliable Is It?
When users upload images that are meant to be viewed and labeled by others, any inappropriate content can directly impact their experience. Ensuring that images are safe and appropriate isn’t just a matter of policy, it’s essential to protect the community and maintain a trustworthy environment. AI-powered models, like those provided by Google Cloud Vision (GCV), aim to help detect sensitive material such as adult or violent imagery.
This post doesn’t focus on the technical implementation details. Instead, it provides a critical analysis of GCV’s performance in a real-world scenario, including its technical limitations, actual test results, and overall reliability.
In our application, users upload image datasets that are later shown to protocol workers for labeling and categorization tasks. Since these images are displayed within the platform, it’s essential to ensure that they don’t contain explicit or harmful content. To automate this moderation step, we integrated Google Cloud Vision (GCV) Safe Search Detection as an initial screening tool.
This feature aims to flag the following categories:
One major limitation is that GCV only supports batch processing at scale when the images are stored in GCS. If images are hosted elsewhere—such as Amazon S3 or private servers with signed URLs—they must be sent one by one using synchronous requests, which severely limits scalability and increases processing time.
Even within GCS, the asyncBatchAnnotateImages API only allows processing up to 100 images per request. For large datasets, this requires additional logic to divide images into batches and coordinate their processing and results collection, adding complexity to the system.
To test the reliability of GCV, we ran an experiment using part of a public dataset containing images of mice. The full dataset includes 2,000 images, from which we selected a subset of 500 images.
To simulate realistic threats, we intentionally injected a few altered images containing sexually explicit or violent content into the subset. This allowed us to measure both:
The altered images containing explicit or violent scenes were successfully detected by GCV. The API correctly returned flags such as VERY_LIKELY or LIKELY under the adult and violence categories for the injected content.
However, the system also produced 15 false positives—images that were completely safe (normal pictures of mice) but were flagged by GCV. In many of these cases, the images were labeled as violence , racy or adult for no valid reason.
Google Cloud Vision can be a helpful first layer of automated moderation, especially when processing large volumes of images. Its integration with GCS enables asynchronous batch processing, and the Safe Search API is straightforward to use.
However, it is not reliable enough to serve as a standalone moderation solution:
In practice, human annotation remains more accurate and, in many cases, more cost-effective, particularly when high precision is required
