Text Analytics Toolbox

Analyze and model text data

Text Analytics Toolbox™ provides algorithms and visualizations for preprocessing, analyzing, and modeling text data. Models created with the toolbox can be used in applications such as sentiment analysis, predictive maintenance, and topic modeling.

Text Analytics Toolbox includes tools for processing raw text from sources such as equipment logs, news feeds, surveys, operator reports, and social media. You can extract text from popular file formats, preprocess raw text, extract individual words, convert text into numerical representations, and build statistical models.

Using machine learning techniques such as LSA, LDA, and word embeddings, you can find clusters and create features from high-dimensional text datasets. Features created with Text Analytics Toolbox can be combined with features from other data sources to build machine learning models that take advantage of textual, numeric, and other types of data.

What Is Text Analytics Toolbox?

Documentation

Getting Started with Text Analytics in MATLAB

Download white paper

Import and Visualize Text Data

Extract text data from sources such as social media, news feeds, equipment logs, reports, and surveys.

Extract Text Data

Import text data into MATLAB^® from single files or large collections of files, including PDF, HTML, and Microsoft^® Word^® and Excel^® files.

Extract Text Data from Files

Parse HTML and Extract Text Content

Analyze Text Data Containing Emojis

Extracting text from a collection of Microsoft Word documents.

Visualize Text

Visually explore text datasets using word clouds and text scatter plots.

Visualize Text Data Using Word Clouds

Visualize Word Embeddings Using Text Scatter Plots

Word cloud showing the relative frequency of words using font size and color.

Language Support

Text Analytics Toolbox provides language specific preprocessing capabilities for English, Japanese, German, and Korean. Most functions also work with text in other languages.

Language Support

Analyze Japanese Text Data

Detect Language of Text

Analyze German Text Data

Import, prepare, and analyze Japanese text.

Preprocess Text Data

Extract meaningful words from raw text.

Clean Text Data

Apply high-level filtering functions to remove extraneous content such as URLs, HTML tags, and punctuations, and correct spellings.

Prepare Text Data for Analysis

Erase Punctuation from Text and Documents

Erase HTTP and HTTPS URLs from Text

Correct spelling in documents

Simplify raw text to work with the most meaningful words.

Simplify raw text (left) to work with the most meaningful words (right).

Filter Stop Words and Normalize Words to Root Form

Prioritize meaningful text data in your analysis by filtering out common words, words that appear too frequently or infrequently, and very long or very short words. Reduce the vocabulary and focus on the broader sense or sentiment of a document by stemming words to their root form or lemmatizing them to their dictionary form.

Remove Stop Words from Documents

Stem or Lemmatize Words

Stemming

Lemmatization

Removing stop words like “a” and “of” from documents.

Identify Tokens, Sentences, and Parts-of-Speech

Automatically split raw text into a collection of words using a tokenization algorithm. Add sentence boundaries, part-of-speech details, and other relevant information for context.

Split Text into Words via Tokenization

Detect the Sentence Boundaries in Documents

Add Part-of-Speech Tags to Documents

Adding part-of-speech and sentence details to tokenized documents.

Convert Text to Numeric Formats

Convert text data to numeric form for use in machine learning and deep learning.

Word and N-Gram Counting

Calculate word frequency statistics to represent text data numerically.

Analyze Text Data Using Multiword Phrases

Term Frequency–Inverse Document Frequency (tf-idf) Matrix

Identify and visualize the most frequently occurring words in a model.

Word Embedding and Encoding

Train word-embedding models such as word2vec continuous bag-of-words (CBOW) and skip-gram models. Import pretrained models including fastText and GloVe.

Visualize Word Embeddings Using Text Scatter Plots

Pretrained FastText Word Embedding

Map Word to Embedding Vector

Bag-of-Words (BoW)

Visualize clusters in a text scatter plot using word embedding.

Machine Learning with Text Data

Perform topic modeling, sentiment analysis, classification, dimensionality reduction, and document summary extraction using machine learning algorithms.

Topic Modeling

Discover and visualize underlying patterns, trends, and complex relationships in large sets of text data using machine learning algorithms such as latent Dirichlet allocation (LDA) and latent semantic analysis (LSA).

Analyze Text Data Using Topic Models

Choose Number of Topics for LDA Model

Compare LDA Solvers

Identifying topics in storm report data.

Document Summarization and Keyword Extraction

Extract summary and relevant keywords from one or more documents automatically and evaluate similarity and importance of documents.

Extract Summary from Documents

Extract Keywords from Text Data Using TextRank

Document Similarity with BM25 Algorithm

Document Scoring with TextRank Algorithm

Extract summary from text.

Sentiment Analysis

Identify the attitudes and opinions expressed in text data to categorize statements as being positive, neutral, or negative. Build models that can predict sentiment in real time.

Analyze Sentiment in Text

Train a Sentiment Classifier

Generate Domain Specific Sentiment Lexicon

Identifying words that predict positive and negative sentiment.

Deep Learning with Text Data

Perform sentiment analysis, classification, summarization, and text generation using deep learning algorithms.

Transformer Models

Leverage transformer models such as BERT, FinBERT, and GPT-2 to perform transfer learning with text data for tasks such as sentiment analysis, classification, and summarization.

Transformer Models for MATLAB