I upload images of places, wrestlers, and general objects to the application. When uploading images of people, I want the Gemini model to generate alt text about their appearance, posture, and attire. When uploading images of places, I want the Gemini model to generate alt text with their official names, coordinates, and locations. To support this, I installed the @mediapipe/tasks-decision library, which uses the EmbeddingGemma 2 model ( embeddinggemma-2-text-vision-440m-litert-lm ) to classify the category of a picture ( people , general , or place ). The category decides which Firebase AI Logic GenerativeModel and prompt generate the text response.
Architecture & Mental Model
Local Model Cache & Classification : When a user uploads an image for analysis, ImageClassifierService checks the browser's CacheStorage ( decision-model-v1 ) for the EmbeddingGemma 2 model, downloads and caches it via HttpClient on a cache miss, and uses DecisionMaker with the provided choices to determine the image's category (…