- Natural Language Processing (NLP) is the use of AI to allow computers to understand people.
- A chatbot is an AI system that communicates with people, say by voice or online chat.
- While there have been great strides in NLP, there is much work to be done. Just some of the challenges include ambiguity of language, nonverbal cues, different dialects and accents, and changes to the language.
- The two main steps with NLP include cleaning/preprocessing the text and using AI to understand and generate language.
- Tokenization is where text is parsed and segmented into various parts.
- With normalization, text is converted into a form that makes it easier for analysis, such as by removing punctuation or contractions.
- Stemming describes the process of reducing a word to its root (or lemma), such as by removing affixes and suffixes.
- Similar to stemming, lemmatization involves finding similar root words.
- For NLP to understand language, there are a variety of approaches like tagging parts of speech (putting the text in the grammatical form), chunking (processing text in phrases), and topic modelling (finding hidden patterns and clusters).
- A phoneme is the most basic unit of sound in a language.
Leave a Reply