An Inside Update on Natural Language Processing
S.Grimes (23 Jul 2016)
Paper’s reference in the IEEE style?
S. Grimes, “An Inside Update on Natural Language Processing,” Breakthrough Analysis, 23-Jun-2016. .
How did you find the paper?
If applicable, write a list of the search terms you used.
- Natural language processing (2001-2016)
Was the paper peer reviewed? Explain how you found out.
No, this is a summary of an interview with Jason Baldridge, a computational linguist.
Does the author(s) work in a university or a government-funded research institute? If so, which university or research institute? If not, where do they work?
The interviewee is an Associate processor in the Department of Linguistics at the University of Texas at Austin and the Co-Founder of "People Pattern", a social media an d public commentary analytics startup ((http://www.jasonbaldridge.com/))
What does this tell you about their expertise? Are they an expert in the topic area?
The interviewee is an expert in NLP
What was the paper about?
The article is an interview with Jason Baldridge and his view on the current state of NLP.
- AI is a great set of tools, but it isn't magic and wont necessarily solve every problem thrown at them.
- Humans learn language from a surprisingly small amount of data. We still need to learn how to do that with AI and we need to figure out architectures, reward mechanisms etc for very deep networks to enable them to generalise information and improve learning
- Interesting trends include
- vectorisation or words and phrases learned on a large, unlabelled corpa. Simlar words will have similar vectors. Bag-of-words techniques treat all words differently and don't learn the relationship between them.
- the use of CNNs for sentiment analysis
- Reinforcement learning (Alpha Go)
- Faster computers, improved sensors
- combination of systems using neural networks as well as interesting language representations (e.g. LSTMs using character level word representations)
- Translation problems (e.g. differences between languages).
- Consider a sequence of sentences like “A garota estava feliz. Foi a praia. Falou come seus amigos.” This sequence should translate as “The girl was happy. [She] went to the beach. [She] talked with her friends.” However, Google Translate returns “The girl was happy. Went to the beach. He talked with his friends.” Part of the reason for this is that there is a bias toward male pronouns in the English language text that the models were trained on, and when the pronoun “ela” is missing from the sentence “falou com seus amigos,” it erroneously fills the subject in the English sentence with “he.”
- Machine learning has dominated NLP since the end of the 1990s, starting with Bell Labs using Hidden Markov Models to recognize speech in the early 1980s; their successes there led to text-oriented NLP work incorporating machine learning starting in the late 1980s. (Philip Resnik calls this DARPA’s shotgun wedding for NLP.)
- A majority of early 2000s work was using discrimitive models like SVM, logistic regression etc. Baysian models became popular in the mid 2000s, now its deep learning
- "The ideal scenario, from my perspective, is that new architectures will be inspired and informed by work in computational linguistics (and linguistics proper), but also that they will be learned without requiring costly and imperfect labeled data. ...But you still need the theory/representations and a way to represent them in your network/algorithm."
If applicable, is this paper similar to other papers you have read for this assignment? If so, which papers and why?
This paper is similar to that written by K.S Jones (Natural language processing: A historical review) but provides a more current overview of natural language processing.
If applicable, is this paper different to other papers you have read for this assignment? If so, which papers and why?
What do these similarities and differences suggest? What are your observations? Do you have any new ideas? Do you have any conclusions?
This paper provides current overview of natural language processing from a current expert. It provides insight around the developments in NLP, largely since the turn of the century, and commentary around likely research directions and issues in the near future.
This question is to be answered after your critical analysis is completed. Which sections (if any) of your critical analysis was this paper cited in?
This paper was referenced in the introduction of the critical analysis and was used to extend the history of NLP beyond the time of the paper written by K.S Jones.