Member-only story
The Decade of Natural Language Understanding and Processing — A Non-Technical Review
Natural Language Understanding (NLU) and Natural Language Processing (NLP) have come a long way in the last decade or so. Thanks to the recent developments, its on the verge of playing a bigger role in our lives. This post is a manager’s guide — not technical background needed. If you are a product manager, program manager with interest in ML or you lead any technology projects with language/text processing — this introductory post is for you.
Language understanding is considered one of the hardest problems for ML and AI. Language is way more complex than it may seem — it is the reflection of a specific culture and a specific context [1]. Linguists have been studying the science of how humans express themselves and what does that tell us about our brains and thoughts. The current ML/AI phase for NLP is closer to engineering than fundamental R&D. The deep learning techniques get us language solution without really understanding human thinking and context but by learning and looping over text repeatedly. We are still far from a real Cognitive solution as some companies claim in their marketing materials.
I’ve been working on NLP for about a decade. Before then I had mainly analyzed structured and unstructured numerical data. The first NLP challenge that my team and I took on was…
