- - 8 min read
Evolution of Natural Language Processing
- - posted on March 04, 2019
Introduction to Natural Language Processing
Natural Language Processing (NLP) a subset technique of Artificial Intelligence which is used to narrow the communication gap between the Computer and Human. It is originated from the idea of Machine Translation (MT) which came to existence during the second world war.
The primary idea was to convert one human language to another human language, for example, turning the Russian language to English language using the brain of the Computers but after that, the thought of conversion of human language to computer language and vice-versa emerged, so that communication with the machine became easy.
In simple words, a language can be understood as a group of rules or symbol. These symbols are integrated and then used for transmitting as well as broadcasting the information. Here rules are applied to suppress the symbols. The area of Natural Language Processing is divided into sub-areas, i.e., Natural Language Generation and Natural Language Understanding which are as the name suggest associated with the generation and understanding the text. The following chart broadly shows these points
Don't get confused by these new terms such as Phonology, Pragmatics, Morphology, Syntax, and Semantics. Let's explore these in a very brief manner -
Phonology - This science helps to deal with patterns present in the sound and speeches related to the sound as a physical entity.
Pragmatics - This science studies the different uses of language.
Morphology - This science deals with the structure of the words and the systematic relations between them.
Syntax - This science deal with the structure of the sentences.
Semantics - This science deals with the literal meaning of the words, phrases as well as sentences.
History of Natural Language Processing (NLP)
As stated above the idea had emerged from the need for Machine Translation in the 1940s. Then the original language was English and Russian. But the use of other words such as Chinese also came into existence in the initial period of the 1960s. Then a lousy era came for MT/NLP during 1966, this fact was supported by a report of ALPAC, according to which MT/NLP almost died because the research in this area did not have the pace at that time. This condition became better again in the 1980s when the product related to MT/NLP started providing some results to customers.
After reaching in dying state in the 1960s, the NLP/MT got a new life when the idea and need of Artificial Intelligence emerged. LUNAR is developed in 1978 by W.A woods; it could analyze, compare and evaluate the chemical data on a lunar rock and soil composition that was accumulating as a result of Apollo moon missions and can answer the related question.
In the 1980s the area of computational grammar became a very active field of research which was linked with the science of reasoning for meaning and considering the user ‘s beliefs and intentions.
In the period of 1990s, the pace of growth of NLP/MT increased. Grammars, tools and Practical resources related to NLP/MT became available with the parsers. The research on the core and futuristic topics such as word sense disambiguation and statistically colored NLP, the work on the lexicon got a direction of research. This quest of the emergence of NLP was joined by other essential topics such as statistical language processing, Information Extraction and automatic summarising.
The discussion on the history of NLP cannot be considered complete without the mention of the ELIZA, a chatbot program which was developed from 1964 to 1966 at the Artificial Intelligence Laboratory of MIT. It was created by Joseph Weizenbaum. It was a program which was based on script named as DOCTOR which was arranged to Rogerian Psychotherapist and used rules, to response the questions of the users which were psychometric-based. It was one of the chatbots which were capable of taking the Turing test at that time.
Recent Trends in Natural Language Processing (NLP )
Nowadays everybody wants the machine to talk, and the only way by which a computer can speak is through Natural Language Processing (NLP). Take the example of Alexa, a conversational product by Amazon. A query is passed to it by the medium of voice, and it can reply by the same medium, i.e., voice. It can be used to ask anything, search for anything, for playing songs or even for cab booking. It seems to be magic, but it is not because of any magic spell, see the below diagram.
This simple diagram is the demonstration of the procedure of Natural Language Processing (NLP) in Alexa. Alexa is not a single example, and these talking machines which are popularly known as Chatbot can even manage complicated interactions and the processes related to the streamlined business using NLP only. In the past chatbot were utilized for only customer interaction with limited capabilities of conversation because they were generally rule-based but after the emergence of Natural Language processing and its integration with Machine Learning and Deep Learning, now chatbot can handle many different areas such Human Resources and Health.
This is not the only use case of NLP where it emerges as a game changer; there are other examples also. Let’s have a brief look at them.
Below is the description of some use cases which shows the power of NLP in the present era.
NLP in health care - Amazon Comprehend Medical services which are used to extract the disease conditions, can handle meditations sessions and can monitor the results of the treatment using clinical trial reports, electronic health records and using patient notes. This is an example of NLP in health analytics where using NLP the prediction of different diseases is possible using pattern recognition methods and patient ‘s speech and their electronic health record.
Sentiment Analysis using NLP - The companies and organizations are now concentrating on the different ways to know their customers so that a personalized touch can be provided. Using sentiment analysis (which is possible only using NLP) the sentiments behind the words can be determined. The sentiment analysis has the capabilities to offer a lot of knowledge about the customer's behavior and their choices which can be considered as significant decision drivers.
Cognitive Analytics and NLP - This is the best example of the collaboration of different technologies, but both come under the same roof of Artificial Intelligence. Using NLP, the conversational frameworks are possible which can take commands by the medium of voice or by the medium of text. Using cognitive analytics, the automation of different technical processes are possible now such generation of a technical ticket related to a technical issue and also handling it in automated or semi-automated ways. The collaboration of these techniques can result in an automated process of handling technical issues inside an organization or providing the solution of some technical problems to the customer also in an automated manner.
Spam Detection - The giants of the technical world such as Google and Yahoo use NLP to classify and filter out the emails which are suspected to be spam. This process is known as Spam Detection and Spam Filtering respectively. It results in an automated process which can classify the email as spam and stop it for entering the Inbox.
NLP in Recruitment - NLP can also be used in both search and selection phases of Job Recruitment, in fact, the chatbot can also be used to handle the job-related query at Initial level which also includes identifying the required skills for a specific job and handling initial level tests and exams.
Conversational Framework - This technology and the devices related to it are gaining so much popularity these days. Alexa which was illustrated above is one of them, but there are Apple's Siri and Google‘s Ok Google which are the examples of the same sort of technology use cases.
Future of Natural Language Processing (NLP)
Considering the market scenario in the case of NLP. The buzz of NLP in the market is growing in an exponential manner which is expected to touch the mark of $ 16 billion by 2021 with the compound growth rate of 16 % annually. The reason behind this growth is rising of the chatbots, urge of discovering the customer insights, transfer of technology of messaging from manual to automated and many other tasks which are required to be automated and involve language/Speech at some point.
Though, as stated above the functionality of NLP revolves around language/speech which refers to words in its basic raw form. No matter what is the medium of the communication, whether it is verbal or written, words are the basic fundamental unit of the functionality of NLP. But in the current NLP, there seems to be a difference in the performance of NLP, when it is handling texts and when it is handling voice. This challenge is going to be addressed in the near future surely. Let’s consider the different scenarios concerning NLP and future.Evolving from human-computer interaction to human-computer conversation.
In the case of interaction only, it is possible to use a single medium which can be anyone verbal or nonverbal communication. But for the communication, it is a necessity to use both medium, verbal and non-verbal together. Though there is a belief that with the development in Natural Language Processing and Biometrics, machines like humanoid robots will acquire the capability to read the expressions of the faces as well as body languages and words also. To accomplish it, there is a necessity of the integration of many modern-day technologies such as recognition of Human users, Sentiment analysis, recommendation analysis and techniques with the engagement in conversations is possible in a dynamic manner.The first critical part of NLP Advancements - Biometrics
The area of nonverbal communications includes body language, touch, gestures, and facial expressions. So to bring nonverbal type communication into the game, there is a necessity of using biometrics like facial recognition, fingerprint scanner, and retina scanner. Nowadays the use of this biometrics are also becoming the prime feature in the field of providing the security on laptops, tablets or even on smartphones which also give a belief to use biometrics to find the patterns in the human’ s facial expressions to recognize the sentiments and emotions from it. Just like different words are used to constitute a whole sentence, different micro expressions are also used to show the feelings in a conversation. These micro-expressions are keys to identifying the difference between different sentiments and emotions and if it is possible to couple them Natural language processing units than this integration can unlock the whole new level of interactions which results human to engaging with the machine using communication The second critical part of NLP advancements - Humanoid Robotics.
Every soul needs a body to express itself. In the same manner, there is a need for a physical unit to convey the NLP advancement in a proper and commercial environment. Thought devices like iPads, interactive TV, dedicated conversational devices (like Siri and Google home) started to cover this domain, but still this is only a scratch on the surface because these have limitations such as use of a specific range of senses (hear, speak, to an extent see but cannot feel touch).
This interaction should be bidirectional as well as the fourth sense (touch) should be included in it; for example, a person is chatting with another person face to face.
Humanoid robots are the necessity of this kind of communication as this can be the body to a programmed artificial soul. As the growth of NLP and Biometrics is gaining pace and accuracy as well, these technologies can give a whole new level to the research of Humanoid robots so that they can express themselves through movement, postures, and expressions.