Natural Language Processing

Table of Contents

Books

Speech and Language Processing 2nd Ed (Daniel Jurafsky, James H. Martin)

One of the most widely referenced and recommended NLP books, written by Stanford University professor Dan Jurafsky and University of Colorado professor James Martin, provides a deep-dive guide on the subject of language processing. It’s intended to accompany undergraduate or advanced graduate courses in NLP or Computational Linguistics. However, it’s a must-read for anyone diving into the theory and application of language processing as they grow and strengthen their analytics capabilities.

It is one of the most sought after and famously referenced and recommended books in the field of NLP. It is authored by Professor Dan Jurafsky from Stanford and James Martin from the University of Colorado. It will give you a deep understanding of the subject of natural language processing. It’s focused on supporting undergraduate or master’s courses in NLP or Computational Linguistics. It is considered as a must-read for those who are diving into the theory and it’s application part of language processing as they rise towards acquiring the skills necessary to strengthen their analytics abilities. A deep focus on web-based language approach, distinct field merging phone-based systems for dialogues, etc. Practical applications are emphasized with technical evaluation. This second edition is much more goal-oriented and can be considered as an extended version of the previous book.

This book provides coverage of NLP from both speech and text perspectives with a strong focus on applications (one in each chapter).

Coverage of the topic feels exhaustive.

Book Contents:

  • Introduction
  • Regular Expressions and Automata
  • Words and Transducers
  • N-grams
  • Part-of-Speech Tagging
  • Hidden Markov and Maximum Entropy Models
  • Phonetics
  • Speech Synthesis
  • Automatic Speech Recognition
  • Speech Recognition: Advanced Topics
  • Computational Phonology
  • Formal Grammars of English
  • Syntactic Parsing
  • Statistical Parsing
  • Features and Unification
  • Language and Complexity
  • The Representation of Meaning
  • Computational Semantics
  • Lexical Semantics
  • Computational Lexical Semantics
  • Computational Discourse
  • Information Extraction
  • Question Answering and Summarization
  • Dialog and Conversational Agents
  • Machine Translation

Natural Language Understanding 2nd Ed (James Allen)

This book is another introductory guide to NLP and considered a classic. While it was published in 1994, it’s highly relevant to today’s discussions and analytics activities and lauded by generations of NLP researchers and educators. It introduces major techniques and concepts required to build NLP systems, and goes into the background and theory of each without overwhelming readers in technical jargon.

This book is a classic material on this subject of NLP. This is a revision of the original book that offers a comprehensive introductory information to natural language understanding with the latest research and developments in the field today. When compared with the first edition of superior understanding and subjective foundation, the new edition provides the learners and readers with the same kind of balanced description of syntax, semantics, and discourse, and concentrates on a uniform framework determined on feature-based context-free grammar and chart Parsers utilized for semantic and syntactic processing. Extensive treatment of problems and issues in discourse and context-dependent explanation is also given.

Handbook of Natural Language Processing (Indurkhya, Damerau)

This comprehensive, modern “Handbook of Natural Language Processing” offers tools and techniques for developing and implementing practical NLP in computer systems. There are three sections to the book: classical techniques (including symbolic and empirical approaches), statistical approaches in NLP, and multiple applications—from information visualization to ontology construction and biomedical text mining.

The second edition has a multilingual scope, accommodating European and Asian languages besides English, plus there’s greater emphasis on statistical approaches. Furthermore, it features a new applications section discussing emerging areas such as sentiment analysis. It’s a great start to learn how to apply NLP to computer systems.

The Handbook of Computational Linguistics and Natural Language Processing (Clark, Fox, Lappin)

Similar to the “Handbook of Natural Language Processing,” this book includes an overview of concepts, methodologies, and applications in NLP and Computational Linguistics, presented in an accessible, easy-to-understand way. It features an introduction to major theoretical issues and the central engineering applications that NLP work has produced to drive the discipline forward. Theories and applications work hand in hand to show the relationship in language research as noted by top NLP researchers. It’s a great resource for NLP students and engineers developing NLP applications in labs at software companies.

The Oxford Handbook of Computational Linguistics (Mitkov)

This handbook describes major concepts, methods, and applications in computational linguistics in a way that undergraduates and non-specialists can comprehend. It’s a state-of-the-art reference to one of the most active and productive fields in linguistics. A wide range of linguists and researchers in fields such as informatics, artificial intelligence, language engineering, and cognitive science will find it interesting and practical. It begins with linguistic fundamentals, followed by an overview of current tasks, techniques, and tools in Natural Language Processing that target more experienced computational language researchers.

Foundations of Statistical Natural Language Processing (Manning, Schuetze)

Another book that hails from Stanford educators, this one is written by Jurafsky’s colleague, Christopher Manning. They’ve taught the popular NLP introductory course at Stanford. Manning’s co-author is a professor of Computational Linguistics at the German Ludwig-Maximilians-Universität.

The book provides an introduction to statistical methods for NLP and a decent foundation to comprehend new NLP methods and support the creation of NLP tools. Mathematical and linguistic foundations, plus statistical methods, are equally represented in a way that supports readers in creating language processing applications.

This foundational book is the first detailed oriented introductory book of statistical natural language processing (NLP) to occur in the market. This book has all the theory and algorithms required for developing NLP tools. Statistical natural-language processing is, one of the most fast-moving and exciting branches of computer science nowadays. Learners or Practitioners and students who want to know this field would be advised to buy this book. It can be called as the most well-thumbed books to be considered for this topic. It gives a broad and deep coverage of mathematical and linguistic basics, as well as in-detail discussions of statistical methods, allowing all the students, learners, and researchers to construct their implementations. The book details the concepts of collocation findings, probabilistic parsing, data retrieval, and other applications.

This book provides an introduction to statistical methods for natural language processing covering both the required linguistics and the newer (at the time, circa 1999) statistical methods.

This book provides a strong foundation to better grasp the newer methods and encodings.

Book contents:

  • Introduction
  • Mathematical Foundations
  • Linguistic Essentials
  • Corpus-Based Work
  • Collocations
  • Statistical Inference: n-gram Models over Sparse Data
  • Word Sense Disambiguation
  • Lexical Acquisition
  • Markov Models
  • Part-of-Speech Tagging
  • Probabilistic Context Free Grammars
  • Probabilistic Parsing
  • Statistical Alignment and Machine Translation
  • Clustering
  • Topics in Information Retrieval
  • Text Categorization

Big Data Analytics Methods: Modern Analytics Techniques for the 21st Century: The Data Scientist’s Manual to Data Mining, Deep Learning & Natural Language Processing (Ghavami)

Peter’s book might seem daunting to a NLP newcomer, but it’s useful as a comprehensive manual for those familiar with NLP and how big data relates in today’s world. It also works as a helpful reference for data scientists, analysts, business managers, and Business Intelligence practitioners. With more than a hundred analytics techniques and methods included, we think this will be a favorite for seasoned analytics practitioners.

Chapters cover everything from machine learning to predictive modeling and cluster analysis. Data science topics including data visualization, prediction, and regression analysis, plus NLP-related fields such as neural networks, deep learning, and artificial intelligence are also discussed. These come with a broad explanation, but Peter goes into more detail about terminology and mathematical foundations, too.

Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit (Bird, Klein, Loper)

This book is a helpful introduction to the NLP field with a focus on programming. If you want have a practical source on your shelf or desk, whether you’re a NLP beginner, computational linguist or AI developer, it contains hundreds of fully-worked examples and graded exercises that bring NLP to life. It can be used for individual study, as a course textbook when studying NLP or computational linguistics, or in complement with artificial intelligence, text mining, or corpus linguistics courses.

Curious about Python programming language? It will walk you through creating Python programs that parse unstructured data like language and recommends downloading Python and the Natural Language Toolkit. On a companion site, the authors have actually updated the book to work with Python 3 and NLTK 3.

A great starting spot for learning the practical basics of natural language processing from the point of view of the Python ecosystem. Also known as the NLTK Book, Natural Language Processing with Python leans heavily on the NLTK library throughout, which is a useful piece of software for learning purposes.

From the book's preface:

  • This book provides a highly accessible introduction to the field of NLP.
  • It can be used for individual study or as the textbook for a course on natural language processing or computational linguistics, or as a supplement to courses in artificial intelligence, text mining, or corpus linguistics. The book is intensely practical, containing hundreds of fully-worked examples and graded exercises.
  • [...]
  • This book is intended for a diverse range of people who want to learn how to write programs that analyze written language, regardless of previous programming experience.

The book is definitely of a practical nature. While you will assuredly have concepts explained as you go, there is little doubt that the book is crafted as a launchpad for those looking to get going with implementing NLP solutions with Python, and doing so now.

This is one of the Best book for Natural Language Processing. This book will give you introduction to Natural Language Processing using Python and Python NLTK Library.

With the help of this book, you will learn how to write Python programs that work with large collections of unstructured text.

Book contents:

  • Language Processing and Python
  • Accessing Text Corpora and Lexical Resources
  • Processing Raw Text
  • Writing Structured Programs
  • Categorizing and Tagging Words
  • Learning to Classify Text
  • Extracting Information from Text
  • Analyzing Sentence Structure
  • Building Feature-Based GRammars
  • Analyzing the Meaning of Sentences
  • Managing Linguistic Data

Natural Language Processing with PyTorch (Rao, McMahan)

This book moves on from traditional NLP techniques to those using neural networks. A practical approach to the subject, it jumps straight into applying neural network NLP methods using PyTorch.

Directly from the book's website, some of the topics covered include:

  • Explore computational graphs and the supervised learning paradigm
  • Master the basics of the PyTorch optimized tensor manipulation library
  • Get an overview of traditional NLP concepts and methods
  • Learn the basic ideas involved in building neural networks
  • Use embeddings to represent words, sentences, documents, and other features
  • Explore sequence prediction and generate sequence-to-sequence models
  • Learn design patterns for building production NLP systems

This is a great starting point for transitioning from more traditional (non-neural network based) NLP techniques to those which have consumed the field in the past few years, which are heavily reliant on deep learning.

Neural Network Methods for Natural Language Processing (Goldberg)

This book is authored by Yoav Goldberg and can be considered only as an Introductory textbook. Neural networks are known as a family of powerful machine learning models. This book sole emphasis is on the application of neural network models to natural language data. The first 50 percent of the book covers all the basics of supervised machine learning and feed-forward neural networks, the fundamentals of working with machine learning over language-based data, and the usage of vector-based data rather than symbolic rendering for words. It also focuses on the computation-graph abstraction, that allows to define and train arbitrary neural networks with ease and is the important basis behind the design of modern neural network software libraries.

Yoav Goldberg writes this book on neural network methods for NLP. You may have begun implementing such methods using the previous book, and while "Natural Language Processing with PyTorch" does a fine job of outlining the intuitions behind its methods, Goldberg's book takes a deeper dive into explaining these concepts without the burden of implementing them in code.

From the book's website:

  • This book focuses on the application of neural network models to natural language data.
  • The first half of the book (Parts I and II) covers the basics of supervised machine learning and feed-forward neural networks, the basics of working with machine learning over language data, and the use of vector-based rather than symbolic representations for words.
  • [...]
  • The second part of the book (Parts III and IV) introduces more specialized neural network architectures, including 1D convolutional neural networks, recurrent neural networks, conditioned-generation models, and attention-based models. These architectures and techniques are the driving force behind state-of-the-art algorithms for machine translation, syntactic parsing, and many other applications.

Firmly in the realm of the theoretical or explanatory, Neural Network Methods for Natural Language Processing will go a long way to shoring up your understanding of how modern neural network based approaches to NLP work, and why they are employed.

This book covers the application of neural network models to natural language processing tasks. In that book, you will learn about basics of supervised machine learning and feed-forward neural networks.

Along with that, you will also learn neural network architectures, including 1D convolutional neural networks, recurrent neural networks, conditioned-generation models, and attention-based models.

Linguistic Fundamentals for Natural Language Processing (Bender)

Of course, flying blind with respect to linguistic fundamentals is not a great idea when working with NLP, and can be of special concern when approaching NLP or computational linguistics from the purely computational side, lacking any formal study in linguistics. This book by Emily M. Bender seeks to help bridge this gap.

The book's website describes the book's purpose as such:

The purpose of this book is to present in a succinct and accessible fashion information about the morphological and syntactic structure of human languages that can be useful in creating more linguistically sophisticated, more language-independent, and thus more successful NLP systems.

Bender backs this up with the following from Chapter 1:

[K]nowledge about linguistic structures can inform the design of features for machine learning approaches to NLP. Put more strongly: knowledge of linguistic structure will lead to the design of better features for machine learning.

The book is organized as 100 individual "essentials" for better understanding morphology and syntax, with the essentials grouped into chapters of related topics. If you do not have a linguistics background (I do not), this book may be a painstaking read (it is supposed to be) but will undoubtedly lead to a better linguistic understanding you can put to use in your NLP career.

Natural Language Processing in Action (Lane, Hapke, Howard)

This book Natural Language Processing in Action is your guide to developing machines that can read and interpret the language used by humans. Within this, you’ll use obtainable Python packages to encapsulate the understanding in text and react to it. This book extends conventional NLP ways to include neural networks, modern deep learning algorithms, and major generative techniques when you tackle real-world issues like extracting names, places and dates, text composing, and free-form answering of listed questionnaires. You will just need a basic understanding of deep learning and intermediate-level Python knowledge. This book also shows how to work with Keras, TensorFlow, Gensim, and SCIkit including scalable pipelines and rule-based NLP. So get this book and Learn all the theory and practical skills necessary to go further by merely understanding the inner functioning of NLP, and start developing your own algorithms and models.

This book is a return to the practical. Covering both traditional and neural network based approaches to NLP, Natural Language Processing in Action could be considered a combination of the first 2 books in this list, covering practical coding solutions using modern tools such as TensorFlow and Keras, among others.

From the book's website:

  • Natural Language Processing in Action is your guide to building machines that can read and interpret human language.
  • In it, you’ll use readily available Python packages to capture the meaning in text and react accordingly.
  • The book expands traditional NLP approaches to include neural networks, modern deep learning algorithms, and generative techniques as you tackle real-world problems like extracting dates and names, composing text, and answering free-form questions.

As a consequence of being the most recently released book in this list (just narrowly edging out Natural Language Processing with PyTorch) as well as that with the most pages, it is likely the most up-to-date and comprehensive practical book in this list, and perhaps even currently available on the market. But that doesn't mean it should be your default choice here either; it depends on the ecosystem you want to work in, and the level of detail you are looking to gain, among other considerations.

As its name suggests “Natural Language Processing in Action”, it’s another best practical approach book for NLP. In that book, you will learn how to build machines that can read and interpret human language.

The authors of this books are experienced NLP engineers. In that book, you will learn how to use readily available Python packages to capture the meaning in text and react accordingly.

Book contents:

  • PART 1 – WORDY MACHINES
    • Packets of thought (NLP overview)
    • Build your vocabulary (word tokenization)
    • Math with words (TF-IDF vectors)
    • Finding meaning in word counts (semantic analysis)
  • PART 2 – DEEPER LEARNING (NEURAL NETWORKS)
    • Baby steps with neural networks (perceptrons and backpropagation)
    • Reasoning with word vectors (Word2vec)
    • Getting words in order with convolutional neural networks (CNNs)
    • Loopy (recurrent) neural networks (RNNs)
    • Improving retention with long short-term memory networks
    • Sequence-to-sequence models and attention
  • PART 3 – GETTING REAL (REAL-WORLD NLP CHALLENGES)
    • Information extraction (named entity extraction and question answering)
    • Getting chatty (dialog engines)
    • Scaling up (optimization, parallelization, and batch processing)

Practical Natural Language Processing: A Comprehensive Guide to Building Real-World NLP Systems (Majumder, Gupta, Surana)

This book provides a comprehensive point-based description of building real-world NLP applications. It will guide you through the process of developing Authentic NLP solutions placed and used in a larger product system. This guide is practical for people who want to develop, manage, and scale NLP systems in a major business environment and modify them for niche industries. You will also learn how to adapt to the various industry verticals such as healthcare, Media, and retail, etc. At the end of this book, you will Understand the wide range of problems, tasks, and different approaches within NLP. Also, you will learn to Perfectly Implement and evaluate different NLP applications machine learning methods. Evaluate all the different algorithms and approaches for NLP product tasks and datasets. Develop and create software solutions by maintaining the best practices during release, deployment, and DevOps for Natural Language Processing systems.

Introduction to Natural Language Processing (Eisenstein)

This book gives a technical direction on the subject of natural language processing―the different ways for developing computer software that understands, produces, and manipulates the language of humans. It emphasizes recent data-driven approaches, attentive towards the techniques from supervised and non-supervised machine learning. Starting with the first section of the book that creates a basic ideation into machine learning by developing a set of tools that can be used throughout the book and application of those for practical word-based textual analysis. This book provides a superb introduction to natural language processing, with the main emphasis on foundational method building and algorithms.

Statistical Machine Translation (Koehn)

This book will give you an introduction to statistical machine translation that is a sub-field of Natural Language Processing.

Book contents:

  • Introduction
  • Words, Sentences, Corpa
  • Probability Theory
  • Word-Based Models
  • Phrase-Based Models
  • Decoding
  • Language Models
  • Evaluation
  • Discriminative Training
  • Integrating Linguistic Information
  • Tree-Based Methods

Statistical Methods for Speech Recognition (Jelinek)

This book will give you an introduction to the topic of statistical speech recognition, another most popular application of Natural Language Processing. In that book, you will find a thorough introduction to speech recognition.

Book contents:

  • The Speech Recognition Problem
  • Hidden Markov Models
  • The Acoustic Model
  • Basic Language Modeling
  • The Viterbi Search
  • Hypothesis Search on a Tree and the Fast Match
  • Elements of Information Theory
  • The Complexity of Tasks – The Quality of Language Models
  • The Expectation-Maximization Algorithm and Its Consequences
  • Decision Trees and Tree Language Models
  • Phonetics from Orthography: Spelling-to-Base Form Mappings
  • Triphones and Allophones
  • Maximum Entropy Probability Estimation and Language Models
  • Tree Applications of Maximum Entropy Estimation to Language Modeling
  • Estimation of Probabilities from Counts and the Back-Off Method

Applied Text Analysis with Python (Bengfort, Bilbro, Ojeda)

In this book, you will learn robust, repeatable, and scalable techniques for text analysis with Python, including contextual and linguistic feature engineering, vectorization, classification, topic modeling, entity resolution, graph analysis, and visual steering.

Along with that, you will also learn how to perform document classification and topic modeling. This book will teach you how to build a dialog framework to enable chatbots and language-driven interaction. Why this Book is Good?

This book will give you a data scientist’s perspective on building language-aware products with applied machine learning techniques.

Courses