Image to Text

Image to Text

With the Image to Text feature, you can describe an image or extract text from an image. The result can be either plain text or a JSON (structured) format.

Describe

The Describe method will generate a new-line delimited list of potential descriptions of the image. This means that each description will be on a new line in the same cell. The Describe method does not extract any text from the image, it only describes what appears in the image.

Extract Text

The Extract Text method will generate a new-line delimited list of text elements detected in the image. For example, if you upload an image of a receipt or a document, this method will pull out the text from that receipt or document. Each element extracted will be on a new line in the same cell.

The Extract Text method will pick up line breaks, capitalization, and punctuation present in the image.

Extract JSON Text

The Extract JSON Text method will output structured data in key-value pairs.

Adding the Image to Text feature

In the Data Editor

1. Open the table where you want to use the Image to Text feature.

2. Add a new Basic Image Column by clicking the (+) plus symbol to the right of the table. This column will store the image you extract text from.

   • New columns added with the plus button will always appear on the right. If you'd like to add a column somewhere else in your data, you can select the dropdown menu on an existing column and choose Add column right. You can also click and drag columns to rearrange them.

3. Add an Image to Text column to store the result. Search for “Image to Text” or navigate to AI → Image to Text.

4. For the Image field, use the image column you created previously.

5. For the Method field, choose if you would like the output to describe the image, extract text from the image in plain format, or extract text in JSON structured format.

6. Note: If the action will be triggered in the Action Editor, then add a basic text Result column where the result will be stored.

7. Click on Done.

In the Layout Editor

1. If you have not yet set up the columns you'll need in the Data Editor, start with the steps above.

2. Add a component that supports actions such as Button, Hint, or Rich Text. Bind this component with the column where the output is stored.

3. In the configuration panel on the right, in the Actions menu, select the Image to Text action.

4. Name the action and choose an icon.

5. For the Image field, select the image column you created previously.

6. For the Method field, choose if you want to describe the image, extract text as JSON, or extract plain text.

7. For the Results Text field, choose the column where you want to store the resulting text.

If you'd like to perform multiple methods on the same image (for example, for describing the image and extracting the image text), you'll need to set up a separate action for each method.

In the Action Editor

1. If you have not yet set up the columns you'll need in the Data Editor, start with the steps above.

2. Select the (+) plus symbol or the + New Action button.

3. Select as the data source for the action the data source of the screen where the action is triggered.

4. Add Image to Text as the first step of the custom action.

5. Configure the Image to Text action: For the Image and Text fields, use the image and text columns created previously.

6. For Method, choose for the output to describe the image, extract text from the image in plain format, or extract text in JSON structured format.

7. Optionally, add a Show Notification action at the end of the custom action to notify the user.