Outreach Sequence Email Sentiment Classification

Purpose:

The purpose of this article is to provide information to Outreach Users regarding the Outreach Insights Sequence Email Sentiment Classification feature.

Intended Audience:

• Outreach Users

Notes:

• This article provides introductory information to Outreach Users/Admins regarding the various features and settings available in the Outreach Platform. Some settings and features may require additional steps for configuration. For more information regarding these topics, refer to the Additional Resources section of this article.

Outreach Insights Sequence Email Sentiment Classification Overview:

Managers often ask, What types of replies are we getting from Prospects?

Previously, managers would spot check email threads to try and diagnose and answer this question --- a daunting and time consuming process that rarely gave insights at scale. Now, with Outreach Buyer Sentiment, email replies are classified as they are received and are surfaced to managers at the altitude that can inform how Prospects are responding to sequences, teams and reps and the entire organization.

Outreach identifies and classifies a sequenced Prospect's email reply as:

• Positive
• Objection
• Referral
• Unsubscribe
• Other

These sentiment categories provide managers more actionable insights into their sequence email performance - beyond open and reply rates. This new layer of insights allows managers to evaluate what drives initial positive outcomes, how well their team handles objections, and successful techniques that turn objections into positive outcomes.

Sentiment Determination:

Outreach built a machine learning model to classify buyer sentiment on the first email reply to a Sequence from Prospects. Therefore, the Prospect has to be in Sequence and reply to an email from a Sequence for the initial reply to be classified in the model.

One -off emails are not counted.

The machine learning model is trained using deep learning methods which understand buyer sentiments and intents from full sentences, and documents much more powerfully than simple keyword-based approaches. 

Supported Languages:

Outreach officially supports English, French, Spanish, and German; however, Outreach has built a sentiment algorithm that supports the 100 most predominant languages. Users will see sentiment predictions for messages in all 100 languages; however, 96 of these are considered to be in beta.

Classifier Accuracy:

The accuracy of the sentiment model ranges between 80-90% for the Outreach officially supported languages. This is for the top-level intents of Positive, Objection, Referral, Unsubscribe, or Other. The accuracy level will continue to improve as Users submit corrections for any inaccurate sentiment predictions. Accuracy for the other 96 unsupported languages varies, some have accuracy on par with Outreach’s officially supported languages.

Outreach came to this accuracy number by withholding a number of tagged emails and tested them against our trained classifier. Outreach developed a custom taxonomy of sentiments for Prospecting based on conversations from thousands of sellers and organizations. Model accuracy may vary from company to company.

Additionally, Outreach will classify sentiment of customers’ new incoming emails and the last 180 days of historical data and provide actionable insights on sequence effectiveness based on sentiment classification of Prospect replies.

Accessing Insights and Reporting:

1. Access the Outreach Platform.
2. Click the Insights and Reporting icon (bar graph) in the navigation sidebar.

By default, Users are presented with the Team Performance Report.

The Team Performance Report provides Users information regarding how Prospects are responding to reps whereas the Sequence Performance Report provides Users with information to evaluate the buyers’ sentiment per sequence level.

For more information regarding the Performance Reports, refer to the applicable articles in the Additional Resources section of this article.

Filters:

Filters available on Team Performance and Sequence Performance reports can be used to update the report data for email sentiment analysis. See details on available filters in corresponding articles of mentioned reports.

Metrics Definitions:

Sentiment classifications are interpreted as described in the table below.

Metrics Drill Down:

Users can click a sentiment to drill further down into pertinent details.

Sentiment drill-downs provide Users the ability to drill into specific email exchanges between a Prospect and a rep. This enables Users to work off of real life examples to better understand the sentiments and replies received by Prospects.

Each sentiment drilldown displays a horizontal bar graph with sub-sentiment replies.

The table of data under the bar graph contains the Prospect name of the replier with seniority and department information, an email preview, and the classified sentiment.

Emails are clickable, providing Users the ability to review email exchanges between a Prospect and a rep. This enables you to bring real life examples to reps when you coach them to show examples of the types of replies they’re eliciting. Additionally, Users can correct the sentiment assigned. Corrected sentiments help train the Outreach machine learning model.

Note: If you change the sentiment in the UI, it will only submit feedback to our model and will neither appear or reflect updates in your data.

Users can access sentiment drill downs by clicking the Total Replies, Positive, Objections, Unsubscribe, Referral, or Other metrics.

Export:

Click Export to download a .CSV file of the reported content.

Additional Resources:

Outreach University - Outreach Reporting

Outreach Insights and Reporting Overview

Outreach Insights and Reporting FAQs

Outreach Insights Sequence Performance Report Overview