Language Processing - Computers that Read and Interpret

Social media has become an integral part of how people interact and communicate. Billions of messages are transmitted daily across multiple platforms, whether posting publicly or messaging directly. Public figures, organisations, and other entities have shared statements and opinions related to various topics and direct public discourse among the general public. Given the vast quantity of comments and messages directed toward social media accounts, it is possible to detect overall sentiment.

There is an opportunity to gauge the sentiment expressed at a macro level via social media. This article will explore artificial intelligence and its application to social media and other sources of textual information that have undertones of sentimentality within them and how to implement it within your organisation.

Solving the Problem via Artificial Intelligence

Given the vast amount of information generated and stored daily, some data points contain extensive insights. It would be impossible for a single person or even billions of people to read it. Current computing technology allows us to store, index and retrieve vast amounts of information. Historically, there have been more difficulties interpreting the data and deriving useful information and insights.

It would be advantageous to process vast amounts of information and have computers scan the data and provide insights. In recent years, various techniques in the text analytics space have emerged, many of which use artificial intelligence/machine learning. Detecting the sentiment in speech requires gold-standard lexicons that provide the context of textual input.

One such approach to determine textual language sentiment is to feed such lexicons into an unsupervised learning algorithm; specifically, VADER (Valence Aware Dictionary and sEntiment Reasoner) uses such an approach. Multiple other methods rely on more traditional machine learning techniques, although the advantage of VADER is that it can operate in near real-time.

Sentiment analysis systems usually produce the following typical outputs when fed textual inputs which give a categorical interpretation of the underlying text:

Positive - Textual content is of a positive undertone and sentimentality
Neutral - Textual content is of a negative undertone and sentimentality
Negative - Textual content is of a negative undertone and sentimentality

It is important to note that people have curated the lexicons over many years, emphasising that high-quality data is required for artificial intelligence systems to perform well and achieve the desired objectives.

Overview of the Organisational Challenge

Given multiple communications endpoints and the size of what organisations operate, it is difficult to track the sentimentality across an organisation. Employees and customers may be spread across different regions, areas or departments where stakeholders and customers express a positive sentimentality while others represent a negative sentimentality. Thus, determining sentimentality across the entire spectrum of an organisation operates is difficult.

It is essential to understand the general organisational perceptions both internally and externally. Traditionally, surveys have performed this function, although these can be biased based on the individuals motivated to fill them in. The use of social media and other data sources can contribute to understanding organisational sentimentality. As an example, a multinational and well-recognised clothing brand such as Adidas, with a presence across multiple countries doing business in various languages across many different cultural backgrounds. Different levels of sentimentality will likely be observed across varied regions. Identifying negative sentiment is a good way of addressing its causes, resulting in more sales and, thus, contributing to the bottom line.

Organisational Data Available as AI Input

The following lists of potential data sources that can be utilised as a means of sentiment analysis:

Social media platforms including Facebook, Twitter, Instagram, TikTok
Customer relationship management systems including Salesforce, Microsoft CRM
Organisational emails and letters sent between employees and customers
Call centre systems such as Cisco and Google Voice are capable of providing transcriptions of conversations
Review aggregators such as Google and Facebook reviews that rank listed businesses between 1-5 stars

Some data issues present themselves when combining information from multiple sources. An obvious problem is how to link customers across various datasets accurately. In this case study, we suggest treating them as independent entities to discover aggregate sentiment across an entire organisation and not targeted at an individual level. Given the length of textual input, data structures must be selected carefully both in the database schema and within the various scripts used to process the data themselves.

Integration Methodology

The following is an overview of the process we would perform at a high level to analyse textual sources within an organisation:

Identify textual references and the systems that capture them, determine which text is a suitable candidate for analysis
Extract textual sources via REST APIs and feed them to a cloud-computing provider for real-time analysis
Run the feeds through Telemus AI™ and return augmented video feeds back to the security monitoring system
Set up customised alerts to the security personnel based upon what is detected

Given that Telemus AI™ takes care of most of the work, the organisation can focus on the business logic and interpreting the results rather than the technical implementation.

Organisational Applications

The following lists other potential applications for your organisation:

Scan online social media endpoints across an organisation and determine the overall sentiment.
Monitor customer notes on CRM systems such as Salesforce to determine the content of the notes.
Scanning emails to gain an overall aggregate metric of the sentiment across an organisation.
Automatically process telephone conversation transcripts and identify both positive and negative customer experiences.
Scanning the feedback left on review aggregator websites as well as the rating.
Automatically detecting antisocial behaviour engaged via digital mediums and countering it before it escalated leading to improved user experience.

As there are many data sources organisations typically have access to, we recommend passing them into a data lake before feeding them into an artificial intelligence system; this will help ensure repeatability and audibility.

Potential and Realised Benefits

A more detailed understanding of the general perceptions of an organisation both within and abroad provides significant benefits in managing relationships proactively. Interpreting and actioning feedback can provide a better product or service to stockholders and customers. In an increasingly digital world, organisations need to understand what their digital footprint says about them. A further understanding of the demographic makeup of where such feedback is arriving can also help improve experiences for all.

Telemus AI™ is an Australian based artificial intelligence company providing advanced solutions to government and enterprise. Contact us today for a free consultation on how the Telemus AI™ can be embedded into your organisation.

Social Media Sentiment Analysis