Natural language processing is a branch of computer science and artificial intelligence about communication with machines using natural language spoken. It is this post’s subject.
Approach
The natural language processing has 3 types of approach: symbolic, statistical and connectionist. In symbolic, uses established rules and lexicons of language to make an analysis. The statistical use example of linguistic phenomena and models based on statistics, adopting mathematical techniques to analyze texts. The connectionist uses both, know the language rules and uses statistical inference.
Processing steps
The steps of natural language processing (NLP) are in this order:
- Phonology: This step is only in if the computer needs to listen to the person, identify the words from sound;
- Morphological analysis: Identify and analyze the word’s structures. It is impractical to have a dictionary with all words, therefore, are implemented rules to regular words and some irregular words inside dominions;
- Lexical analysis: determine the meaning of individual words;
- Syntactic analysis: Recognizes and analyze phrases and gramatical rules. A syntactical tree is used to make the recognition top-down.
Another way of recognition is bottom-up.
- The cat hunted the rat.
- Article cat hunted the rat.
- Article Noun hunted the rat.
- Article Noun Verb the rat.
- Article Noun Verb Article rat.
- Article Noun Verb Article Noun.
- Phrase.
- Semantic analysis: Verify the phrase`s meaning mapping the syntactic structure, needs have information about the real world to eliminate ambiguity and illogical sentences, like “hot ice cream”;
- Discourse analysis: Observe how the sentences relate each other in a text;
- Pragmatic analysis: Verify if the semantic interpretation is correct and determinate non-clear meanings.
Chatbots and other applications
The most notable application of NLP are the chatbots, programs that chat like people.
Online translators also use NLP.
Some other applications:
- Human resources recruitment;
- Search tools like Google and Qwant;
- Spam filter;
- Text corrector;
- Report generator.
