Google Research has released Agile Deliberation, a new human-in-the-loop framework designed to build image classifiers for subjective concepts like healthy food and clickbait. Instead of assuming users already know a concept's boundary, the framework helps them find it and translates vague ideas into structured written definitions that act as policies and text prompts for vision-language models.
The system operates in two main stages by using prompt chaining with Gemini 2.5 Pro and Gemini 2.5 Flash via the API without fine-tuning. Stage one breaks a concept into unit subconcepts, while stage two hunts for semantically borderline images using dictionary learning and a UCB multi-armed bandit, bypassing traditional uncertainty sampling. Evaluated across 18 live sessions with non-professional participants, the framework achieved an average F1 score increase of 10.5 points over zero-shot Gemini 2.5 Flash.
Developers can run the framework using an included Jupyter notebook demo that requires a Gemini API key and a FAISS index built over CLIP embeddings. This tool matters because it provides a reliable, scalable way to define and classify subjective visual concepts for content moderation and enterprise applications.