NLP, Machine Learning: Killer Combo for Unstructured Data - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IT Leadership // Team Building & Staffing
02:00 PM
Connect Directly

NLP, Machine Learning: Killer Combo for Unstructured Data

To apply AI to customer service interactions, you need both machine learning and natural language processing, which is why demand for NLP skills are on the rise in the job market.

If you are looking for the next emerging job title or sought-after skill set that will make companies compete to hire you and pay you a premium, consider gaining some expertise in working with unstructured data and natural language processing.

While both machine learning and natural language processing skills are in high demand in the market today, NLP experts are harder to find.

"NLP, even among analytics, is among the hardest skills in the market," Sameer Gupta told InformationWeek. He is the AI and advanced analytics leader at EY, specializing in the financial services market. Gupta also recently authored a whitepaper about using NLP and ML to harness the insights from unstructured data.

Image: momius -
Image: momius -

It is the combination of NLP and machine learning that will enable organizations to gain insights from unstructured data such as emails, chat transcripts, outbound marketing materials, internal memos, legal documents, and complaint logs, in a way that hasn't been previously possible. Organizations can mine this type of unstructured data for several purposes, including regulatory compliance and sentiment analysis, according to Gupta. While organizations may have gained momentum in efforts to apply machine learning technologies to their structured data, they are still just getting started with NLP, unstructured data, and technologies such as sentiment analysis, he said. That's why there's a high demand for these skills that are still in short supply.

"Some functional and working knowledge of NLP and machine learning is one of the most sought-after skills," Gupta said. He works with companies in the financial services industry, which is a bit ahead of many other industries in terms of applying these emerging technologies but is not as advanced as some of the tech leaders in the space such as Amazon and Facebook.

"Organizations don't know where to start, but that hasn't stopped them from trying or starting," Gupta told InformationWeek. "They recognize there is a lot of insight buried in unstructured data."

For instance, some banks have been around for more than 200 years, and they are competing with digital native startups. Providing superior customer responsiveness is crucial to being competitive against these new market entries. These companies need to be able to do a better job of harvesting data from previous interactions to provide service and messaging that is in sync with what customers want.

NLP and unstructured data can also drive down the costs of regulartory compliance.

"Some of the clients we serve have more than 1,000 people just listening to calls and reading notes and chat transcriptions to be able to see if the interaction meets compliance needs," Gupta said. By automating those processes with NLP and machine learning, organizations can lower the costs of that monitoring.

The NLP market right now, like many new markets, is primarily populated with a lot of startup companies. You won't necessarily find your longtime enterprise software vendor offering this kind of functionality built into their existing products. Yet.

"The days where people could rely on one vendor to provide all the components in the stack are not quite here anymore," Gupta said. "The bigger vendors are responding in two ways -- closing the gap in their overall stack and also making their stack more open and flexible to respond to niche providers and tools.

For companies looking to hire these types of skills in-house, Gupta said there are three levels of skills needed for someone to be an effective contributor. At the base level, there are the modeling skills required -- generally the ability to program in Python. The next level is some knowledge of about a half dozen key NLP techniques. The final level is the ability to combine those first two skill sets and apply them to the business problem. That third level is the key to a successful job candidate, according to Gupta. Having a programming language on your resume doesn't mean you have the experience to know how to apply it in a real-world setting.

Yet hiring the right skills is just one part of a larger effort, Gupta said.

"Scaling all of this in financial services is not just driven by talent," he said. "It's also driven by the right architecture and organization constructs."

To scale it, you must identify the right architecture that enables you to reuse models and efforts.

"The ones who do that will be the most successful."

For more on AI, skills, and careers, read these articles:

Automation, AI Impact: The Future of Work

Data Hunter: The New Sexy Technology Job

AI, Automation Job Trend to Reverse in 2020

Prepping the Enterprise for the AI Apocalypse

Data Science: A Guide to Careers and Team Building

AI & Machine Learning: An Enterprise Guide

Jessica Davis is a Senior Editor at InformationWeek. She covers enterprise IT leadership, careers, artificial intelligence, data and analytics, and enterprise software. She has spent a career covering the intersection of business and technology. Follow her on twitter: ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Why 2021 May Turn Out to be a Great Year for Tech Startups
John Edwards, Technology Journalist & Author,  2/24/2021
How GIS Data Can Help Fix Vaccine Distribution
Jessica Davis, Senior Editor, Enterprise Apps,  2/17/2021
11 Ways DevOps Is Evolving
Lisa Morgan, Freelance Writer,  2/18/2021
White Papers
Register for InformationWeek Newsletters
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you.
Flash Poll