Vision Label Overview

The contents of this page are based on the original Firebase Documentation

With ML Kit's image labeling APIs, you can recognize entities in an image without having to provide any additional contextual metadata, using either an on-device API or a cloud-based API.

Image labeling gives you insight into the content of images. When you use the API, you get a list of the entities that were recognized: people, things, places, activities, and so on. Each label found comes with a score that indicates the confidence the ML model has in its relevance. With this information, you can perform tasks such as automatic metadata generation and content moderation.

This is a beta release of ML Kit for Firebase. This API might be changed in backward-incompatible ways and is not subject to any SLA or deprecation policy.

	On-device	Cloud
Pricing	Free	Free for first 1000 uses of this feature per month: see Pricing
Label coverage	400+ labels that cover the most commonly-found concepts in photos. See below.	10,000+ labels in many categories. See below. Also, try the Cloud Vision API demo to see what labels can be found for an image you provide.
Knowledge Graph entity ID support	✓	✓

Example on-device labels

The device-based API supports 400+ labels, such as the following examples:

Category	Example labels
People	Crowd Selfie Smile
Activities	Dancing Eating Surfing
Things	Car Piano Receipt
Animals	Bird Cat Dog
Plants	Flower Fruit Vegetable
Places	Beach Lake Mountain

Example cloud labels

The cloud-based API supports 10,000+ labels, such as the following examples:

Category	Example labels	Category	Example labels
Arts & entertainment	Sculpture Musical Instrument Dance	Astronomical objects	Comet Galaxy Star
Business & industrial	Restaurant Factory Airline	Colors	Red Green Blue
Design	Floral Pattern Wood Stain	Drink	Coffee Tea Milk
Events	Meeting Picnic Vacation	Fictional characters	Santa Claus Superhero Mythical creature
Food	Casserole Fruit Potato chip	Home & garden	Laundry basket Dishwasher Fountain
Activities	Wedding Dancing Motorsport	Materials	Ceramic Textile Fiber
Media	Newsprint Document Sign	Modes of transport	Aircraft Motorcycle Subway
Occupations	Actor Florist Police	Organisms	Plant Animal Fungus
Organizations	Government Club College	Places	Airport Mountain Tent
Technology	Robot Computer Solar panel	Things	Bicycle Pipe Doll

Google Knowledge Graph entity IDs

In addition the text description of each label that ML Kit returns, it also returns the label's Google Knowledge Graph entity ID. This ID is a string that uniquely identifies the entity represented by the label, and is the same ID used by the Knowledge Graph Search API. You can use this string to identify an entity across languages, and independently of the formatting of the text description.