Extract text from Image and PDF - Using Power Automate and AI Builder

[sandeepstw]

Introduction  

In today's data-driven world, we often need to extract valuable information from various sources, including images and PDFs. Power Automate and AI Builder offer a powerful solution for extracting text from these documents, enabling businesses to streamline processes and gain insights from previously unstructured data. 

Power Automate and AI Builder 

Power Automate is a cloud-based automation platform that allows users to create automated workflows, or flows, to connect various services and applications.  

AI Builder, a component of Power Automate, provides pre-built AI models that can be integrated into flows to perform tasks such as text recognition, sentiment analysis, and object detection. 

Setting Up Text Extraction Automation 

Before diving into automation, ensure that all necessary components are in place. This includes access to Power Automate, AI Builder, and any additional tools or services required for the specific workflow. 

Go to the Power Automation dashboard and click on “Instant Cloud flow”. And select “Manually Triggered flow”. Give the name of flow whatever suits you, I given name “Extract text from images or PDF documents using AI Builder and Power Automate”. 

Then click on add action to add a new action in flow, search for “AI builder” in the “Add new action dialog box”. And click on see all in right side to see all action available in that component.  

In the expanded list you can see all available actions in the AI connector. We need to find the action “Recognize text in an Image or in a PDF document”. 

“Recognize text in an image or in a PDF document” contains a file input parameter. The rest of things will be done by this action itself you do not need to worry about it.  

We need to add a variable to appending text to a string so we can use it further in our flow. I added an action “Initialize a variable” action. I named it “Extract Text.” 

We will add one more variable to add a new line in String. I added a variable “NewLine” variable.  

Next step to add “Apply to each” control and pass response from action “Recognize text in an image or in a PDF document” ai build action.  

Again, we will add “Apply to each” control to fetch each line from the AI response.  

Add new action “Append text to string” under variable control. And then from the name dropdown pick the name “Extract Text” created before in this flow. And in value, input adds values ‘line’ coming from ai response as a new line and “NewLine” variable to add a new line in the string.  

Add new action to fetch profile information from office database ”GetProfile V2”. 

At the end of this flow, we will send an email with extracted data to see the output and send the uploaded file as an attachment. For this, we will add “Send Email V2” from Outlook Connector.  

In Parameter, we are using - 

To – Mail (From Get profile v2 action) 

"emailMessage/To": "@outputs('Get_my_profile_(V2)')?['body/mail']", 

Subject - “Text Recognition result” 

"emailMessage/Subject": "Text Recognition results", 

Body -  

<p><span style=\"font-size: 16px;\">Congratulations </span>@{outputs('Get_my_profile_(V2)')?['body/givenName']}<span style=\"font-size: 16px;\"></span><span style=\"font-size: 16px;\">! Your Power Automate flow ran successfully. Here is all the text extracted from the attached file:</span><span>\n\n</span>@{variables('Extracted text')}<span style=\"font-size: 16px;\"></span><span>\n\n</span><span style=\"font-size: 16px;\">Learn more about AI Builder Text Recognition and the information it can extract: https://go.microsoft.com/fwlink/?linkid=2161167</span></p> 

Attachment -  

"emailMessage/Attachments": [ 

        { 

          "Name": "@triggerBody()?['file']?['name']", 

          "ContentBytes": "@triggerBody()?['file']?['contentBytes']" 

        } 

      ] 

After adding all these values make sure you saved flow.  

Testing - 

Run flow using the flow detail page. It will ask for an image upload. Upload an image. Once flow runs successfully. You will get a mail with data and upload an image. Like this. 

Email body  

Congratulations Sandeep! Your Power Automate flow ran successfully. Here is all the text extracted from the attached file: 
 
Home Marketing Sales v Operations v Productivity v Business Analysis v HR Admin v Master v Settings v Edit Public group & 22 members ... S T W STW365 + New v 6] Promote Translation Page details J Analytics Published 8/27/2023 Share v Swati Chavan is editing this page 6 Edit LEADS DASHBOARD LEADS DASHBOARD T W FY FY 2023-2024 V Industry Source City Sales Person Status All V All V All V All V All V Disqualified Open Qualified 7 123 13 STATUS RECORD SOURCE TYPE 
 
Learn more about AI Builder Text Recognition and the information it can extract: https://go.microsoft.com/fwlink/?linkid=2161167 

According to the uploaded image we will same data in email, we got from power automate flow.  

Hope this will help you to extract data from ID card, pdfs or any scanned document to automate your process in your business using Power Automate and AI builder.  

Happy learning and process automation.  

Thanks!!