Twitter Sentiment Analysis with Azure Cognitive Services

Voice of the Customer Series – Part 2

In part 1 of my voice of the customer series blog post, I demoed how Twitter data can automatically be saved in Google Sheets using Microsoft flow.

In this post, I’ll show how you can leverage the technology from Microsoft Azure Cognitive Services to perform sentiment analysis on the tweets. This will allow you to determine how positively or negatively a brand or topic is being mentioned on Twitter. Continuing on from part 1, the demo will again include Vancouver car sharing companies.

Why Would We Use Sentiment Analysis?

For smaller organizations, sentiment analysis may not be useful for analyzing their social media data. Perhaps you only receive a few tweets per day, and you’re very aware of your brand’s online activity.

But what if you receive hundreds, or even thousands of tweets per day? What if you have multiple social media accounts from a variety of different platforms: Instagram, LinkedIn, Glassdoor, Yelp, etc. You likely don’t have time to read each message, and it would be very difficult to try and flag posts which require extra care.

You also wouldn’t be able to consistently measure how positively or negatively your brand is trending on the internet.

Enter sentiment analysis. By assigning scores to blocks of text, this analysis allows you to measure the sentiment of each tweet/product review/social media mention.

Introducing Azure Cognitive Services: Text Analytics

To demonstrate sentiment analysis in action, I’ve entered a negative message into the Azure text analytics API. It works by using machine learning to assign a 0% (most negative) to 100% (most positive) score based on text input it receives. As expected, my text results in a very low sentiment score of 2%.

 

Cognitive Services Text Analytics

Integrating Azure Cognitive Services API with Power BI

Cognitive Services will be natively integrated into Power BI premium, but not every organization can afford the $5,800+/month price tag. With a little bit of extra work, you can still leverage the machine learning power of Azure Cognitive Services, for free. Using Power BI, we’ll:

  1. Load the Twitter data,
  2. Send a call to the Cognitive Services Text Analytics API, and finally
  3. Return a sentiment score for each tweet.
Twitter to Power BI to Azure Cognitive Services

*Note: this is only proof of concept (POC) demonstration. If building this into an enterprise/production solution, you’d likely want to integrate this API into an existing social media ETL process.

Grabbing API Keys

You’ll first need to create an Azure account and setup resource groups and Azure Cognitive Services from the Azure portal. This setup is beyond the scope of this post, but you can follow along here.

Lucky for us, the free tier of Cognitive Services allows up to 5,000 transactions per month. Once you’ve set the Azure account and services up, you can grab the API keys needed through your Azure Portal:

Grab API Keys

Creating Function to Call API

Next, we’ll go into the Power BI Get Data/Power Query editor to create a function. This will be used to call the API.

Go to the home menu, click Get Data > Blank Query, and then click on the Advanced Editor. Here, you will enter in the below function code (taken from the Microsoft Tutorial):

Function
// Returns the sentiment score of the text, from 0.0 (least favorable) to 1.0 (most favorable)
(text) => let
    apikey      = "YOUR_API_KEY_HERE",
    endpoint    = "https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/sentiment",
    jsontext    = Text.FromBinary(Json.FromValue(Text.Start(Text.Trim(text), 5000))),
    jsonbody    = "{ documents: [ { language: ""en"", id: ""0"", text: " & jsontext & " } ] }",
    bytesbody   = Text.ToBinary(jsonbody),
    headers     = [#"Ocp-Apim-Subscription-Key" = apikey],
    bytesresp   = Web.Contents(endpoint, [Headers=headers, Content=bytesbody]),
    jsonresp    = Json.Document(bytesresp),
    sentiment   = jsonresp[documents]{0}[score]
in  sentiment

We’ll now use this function to create a new column that returns the sentiment scores for each block of text. Go to your text data table, select the text column you would like to assign scores to, and then click Add Column > Invoke Custom Function:

Invoke function Power BI

Next, select your newly created function (SentimentFunction), and apply it to the text column:

Invoke custom function

Once you’ve applied these steps, the sentiment scores will appear as a new column:

New column Power BI

We now have a consistent way to measure how positive or negative each tweet is! If you would like to view the completed Power BI file created for this post, you can download it from my GitHub repository here.

By bringing in the sentiment scores for our Twitter data, we can now perform a great deal of analysis to track how our social media channels are trending over time, and determine who our most unhappy customers are.

In my next post I’ll show how to create visualizations and create an interactive dashboard in Power BI with our Twitter data and sentiment scores.

If you have any questions, feel free to reach out to me on LinkedIn or Twitter.

Leave a Reply