This page explains the type of results returned by the Image analysis BLOCK.

For a detailed example of using the Image analysis BLOCK, refer to Using the Google Cloud Vision API Machine Learning service.

warning The following image analysis results were recorded based on the February 2019 version of Google Cloud Platform | Google Cloud Vision API Documentation | Method: images.annotate open_in_new

Image analysis results

By default, the Image analysis BLOCK exports its results as JSON format data to a variable named _ (underscore). You can configure the name of this variable with the Results storage variable setting in the Advanced settings portion of the property menu.

{
  "faceAnnotations": [{...}],
  "landmarkAnnotations": [{...}],
  "logoAnnotations": [{...}],
  "labelAnnotations": [{...}],
  "textAnnotations": [{...}],
  "fullTextAnnotation": {...},
  "safeSearchAnnotation": {...},
  "imagePropertiesAnnotation": {...},
  "cropHintsAnnotation": {...},
  "gcs_url": string,
  "timestamp": number
}

Other than gcs_url and timestamp, these results are only exported when their corresponding features are detected by the analysis. For example, faceAnnotations will only be returned when a face is detected in the image.

Facial recognition ("faceAnnotations")

Facial recognition results are returned as an array ([...] format). The detected facial data that makes up the array are objects ({...} format).

"faceAnnotations": [
  {...},
  {...}
]

The format of the objects showing facial detection results are explained below. For an example of outputted results, refer to Examples of using the Image analysis BLOCK > Facial recognition example.

{
  "boundingPoly": {...},
  "fdBoundingPoly": {...},
  "landmarks": {...},
  "rollAngle": number,
  "panAngle": number,
  "tiltAngle": number,
  "detectionConfidence": number,
  "landmarkingConfidence": number,
  "joyLikelihood": string,
  "sorrowLikelihood": string,
  "angerLikelihood": string,
  "surpriseLikelihood": string,
  "underExposedLikelihood": string,
  "blurredLikelihood": string,
  "headwearLikelihood": string
}

Landmark recognition ("landmarkAnnotations")

Landmark recognition results are returned as an array. The detected landmark data that makes up the array are objects.

"landmarkAnnotations": [
  {...},
  {...}
]

The format of the objects showing landmark detection results are explained below. For an example of outputted results, refer to Examples of using the Image analysis BLOCK > Landmark recognition example.

{
  "mid": string,
  "description": string,
  "score": number,
  "boundingPoly": {...},
  "locations": [{...}]
}

Logo recognition ("logoAnnotations")

Logo recognition results are returned as an array. The detected logo data that makes up the array are objects.

"logoAnnotations": [
  {...},
  {...}
]

The format of objects showing logo detection results are explained below. For an example of outputted results, refer to Examples of using the Image analysis BLOCK > Logo recognition example .

Object recognition ("labelAnnotations")

Object recognition results are returned as an array. The detected object data that makes up the array is an object.

"labelAnnotations": [
  {...},
  {...}
]

The format of the object showing object detection results is explained below. For an example of outputted results, refer to Examples of using the Image analysis BLOCK > Object recognition example.

{
  "mid": string,
  "description": string,
  "score": number
}

Text recognition ("textAnnotations")

Text recognition results are returned as an array. The text data that makes up the array is an object.

"textAnnotations": [
  {...},
  {...}
]

The format for the results is explained below. For an example of outputted results, refer to Examples of using the Image analysis BLOCK > Text recognition example.

{
  "locale": string,
  "description": string,
  "boundingPoly": {...}
}

OCR (document) detection ("fullTextAnnotation")

The format of OCR (document) detection results data is explained below.

"fullTextAnnotation": {
  "pages": [
    {...},
    {...}
  ],
  "text": string
}

Adult content detection ("safeSearchAnnotation")

The format of the data showing adult content detection results is explained below. For an example of outputted results, refer to Examples of using the Image analysis BLOCK > Adult content detection example.

{
  "adult": string,
  "spoof": string,
  "medical": string,
  "violence": string
}

Color analysis ("imagePropertiesAnnotation")

The format of color analysis results data is as shown below.

"imagePropertiesAnnotation": {
  "dominantColors": {...}
}

The dominantColors object contains information about the colors in the image. Usually, it contains information for several colors.

"dominantColors": {
  "colors": [
    {...},
    {...}
  ]
}

The format of the color information is as follows. For an example of outputted results, refer to Examples of using the Image analysis BLOCK > Color analysis example.

{
  "color": {...},
  "score": number,
  "pixelFraction": number
}

The objects showing the RGB values of the color are formatted as follows:

{
  "red": number,
  "green": number,
  "blue": number
}

Crop hints annotation ("cropHintsAnnotation")

The format for crop hints annotation results is as follows. For an example of outputted results, refer to Examples of using the Image analysis BLOCK > Image properties annotation.

{
  "cropHints": [
    {...},
    {...}
  ]
}

Crop hint ("cropHint")

The format for crop hint results data is as follows:

{
  "boundingPoly": {...},
  "confidence": number,
  "importanceFraction": number
}

Bounding polygon objects ("boundingPoly"/"fdBoundingPoly")

The format for objects that express polygons is as shown below.

{
  "vertices": [{...}, ..., {...}]
}

Vertices object

The vertices object shows two-dimensional coordinates within the image. The data is formatted as follows:

{
  "x": number,
  "y": number
}

Likelihood

Likelihood is shown using one of the six possible strings shown below.

Facial features (landmarks) object

The facial landmarks object shows the type of facial feature (left eye, right eye, etc.) and the feature's position in the image. The data is formatted as shown below.

{
  "type": string,
  "position": {...}
}

Types of features

Facial features are expressed as strings. The various strings and their meanings are as shown below.

Facial feature position object

Three dimensional coordinates showing the position of each facial feature. The data is formatted as follows:

{
  "x": number,
  "y": number,
  "z": number
}

Geographical position object ("locations")

An array that shows the geographical position. The position is expressed as objects for the latitude and longitude.

"locations": [
  {...},
  {...}
]

The data for the latitude and longitude objects is expressed as shown below.

{
  "latLng": {
    "latitude": number,
    "longitude": number
  }
}

Pages ("page")

The format for page data results for text recognition is as follows:

{
  "blocks": [
    {...},
    {...}
  ],
  "width": number,
  "height": number,
  "property": {...}
}

Additional properties (property)

The format of the additional properties data is as follows:

{
  "detectedLanguages": [
    {...},
    {...}
  ],
  "detectedBreak": {...}
}

Detected language ("detectedLanguage")

The format of the detected language results is as follows:

{
  "languageCode": string,
  "confidence": number
}

Detected break ("detectedBreak")

The format of detected break results is as follows:

{
  "type": string,
  "isPrefix": boolean
}

Break type ("breakType")

The types of breaks are as follows:

Block ("block")

A block is a logical element detected on the page. The format for block results data is as follows:

{
  "boundingBox": {...},
  "paragraphs": [
    {...},
    {...}
  ],
  "blockType": string,
  "confidence": number,
}

0----1
|    |
3----2

2----3
|    |
1----0

Paragraph ("paragraph")

The format of paragraph results is as follows:

{
  "boundingBox": {...},
  "words": [
    {...},
    {...}
  ],
  "confidence": number
}

0----1
|    |
3----2

2----3
|    |
1----0

Word ("word")

The format of word data results is as follows:

{
  "property": {...},
  "boundingBox": {...},
  "symbols": [
    {...},
    {...}
  },
  "confidence": number
}

0----1
|    |
3----2

2----3
|    |
1----0

Symbol ("symbol")

The format for symbol data results is as follows:

{
  "property": {...},
  "boundingBox": {...},
  "text": string,
  "confidence": number
}

0----1
|    |
3----2

2----3
|    |
1----0

Block type ("blockType")

The type of block detected by the text recognition. The different possible types are as follows: