big data - words matter

90% of the Big Data that is being generated everyday is not in the form of numbers but words (unstructured data). What people write-click-share-view-upload everyday can reveal real insights about their motivations that can help marketers build relevant and conversion oriented experiences. Specifically emails, reviews, requests, complaints, FAQs from consumers can reveal insights @ scale that can be used by marketers to observe patterns, capture trends and identify high risk events.
 
The process of applying machine learning and natural language processing techniques to text data is complex. Not surprisingly, given that language is about concepts-meaning-inferences-nuance and specificity the process is part science and part art.
 
Lets look at two steps within the entire modeling process that has direct linkage to marketing  (Choosing a modeling technique to run your data, Data Transformation, Test-Training Data etc. are crucial parts of the process) :
 
Step 1 : Feature Extraction
Extraction of meaningful phrases within text, entity extraction (person, places, products and organisations). There are a wide range commercial text analytics tools that help you do this. Some of them are : IBM Watson Alchemy API, Lexalytics, Microsoft Text Azure Analytics API.
 
Step 2 : Codification
After the extraction of key phrases and entities is done the next step is to create tokens (phrases/ entity that will signal a key event has occurred) is basic first step. This step requires an in-depth understanding of category and consumer to help capture the signal Vs the noise. Whats meaningful and actionable for one category may be less so for another. Red flags for a skin care brand could be words like (allergy) while (call drop offs) would be more worrisome for a telco provider.
 
Example A : For an Insurance provider focussed on building a reputation for simple and easy approvals, analysis of consumer reviews-FAQs-call center transcripts might indicate that consumers are dropping-outs because of the confusing application form that they need to fill in. In this case positive/ negative phrases related to application-form filling etc become meaningful.
 
Token Example : Creating a token with negative phrases linked to application become the trigger for a company to respond immediately and appropriately to rectify the situation.
 
Detecting the signals Vs the noise for your brand in ways that are actionable will allow you to effectively improve your brand goals in immediate and measurable ways. Lastly, continuous human intervention is needed as patterns change over time, new vocabulary and contexts emerge.

reference