encode

Encode documents as matrix of word or n-gram counts

Use encode to encode an array of tokenized documents as a matrix of word or n-gram counts according to a bag-of-words or bag-of-n-grams model. To encode documents as vectors or word indices, use a wordEncoding object.

Syntax

counts = encode(bag,documents)
counts = encode(bag,words)
counts = encode(___,Name,Value)

Description

example

counts = encode(bag,documents) returns a matrix of frequency counts for documents based on the bag-of-words or bag-of-n-grams model bag.

example

counts = encode(bag,words) returns a matrix of frequency counts for a list of words.

example

counts = encode(___,Name,Value) specifies additional options using one or more name-value pair arguments.

Examples

collapse all

Encode an array of documents as a matrix of word counts.

documents = tokenizedDocument([
    "an example of a short sentence" 
    "a second short sentence"]);
bag = bagOfWords(documents) 
bag = 
  bagOfWords with properties:

          Counts: [2x7 double]
      Vocabulary: [1x7 string]
        NumWords: 7
    NumDocuments: 2

documents = tokenizedDocument([
    "a new sentence" 
    "a second new sentence"])
documents = 
  2x1 tokenizedDocument:

    3 tokens: a new sentence
    4 tokens: a second new sentence

View the documents encoded as a matrix of word counts. The word "new" does not appear in bag, so it is not counted.

counts = encode(bag,documents);
full(counts)
ans = 2×7

     0     0     0     1     0     1     0
     0     0     0     1     0     1     1

The columns correspond to the vocabulary of the bag-of-words model.

bag.Vocabulary
ans = 1x7 string array
    "an"    "example"    "of"    "a"    "short"    "sentence"    "second"

Encode an array of words as a vector of word counts.

documents = tokenizedDocument([
    "an example of a short sentence"
    "a second short sentence"]);
bag = bagOfWords(documents)
bag = 
  bagOfWords with properties:

          Counts: [2x7 double]
      Vocabulary: [1x7 string]
        NumWords: 7
    NumDocuments: 2

words = ["another" "example" "of" "a" "short" "example" "sentence"];
counts = encode(bag,words)
counts = 
   (1,2)        2
   (1,3)        1
   (1,4)        1
   (1,5)        1
   (1,6)        1

Encode an array of documents as a matrix of word counts with documents in columns.

documents = tokenizedDocument([
    "an example of a short sentence"
    "a second short sentence"]);
bag = bagOfWords(documents)
bag = 
  bagOfWords with properties:

          Counts: [2x7 double]
      Vocabulary: [1x7 string]
        NumWords: 7
    NumDocuments: 2

documents = tokenizedDocument([
    "a new sentence"
    "a second new sentence"])
documents = 
  2x1 tokenizedDocument:

    3 tokens: a new sentence
    4 tokens: a second new sentence

View the documents encoded as a matrix of word counts with documents in columns. The word "new" does not appear in bag, so it is not counted.

counts = encode(bag,documents,'DocumentsIn','columns');
full(counts)
ans = 7×2

     0     0
     0     0
     0     0
     1     1
     0     0
     1     1
     0     1

Input Arguments

collapse all

Input bag-of-words or bag-of-n-grams model, specified as a bagOfWords object or a bagOfNgrams object.

Input documents, specified as a tokenizedDocument array, a string array of words, or a cell array of character vectors. If documents is a string array or a cell array of character vectors, then it must be a row vector representing a single document, where each element is a word.

Tip

To ensure that the documents are encoded correctly, you must preprocess the input documents using the same steps as the documents used to create the input model. For an example showing how to create a function to preprocess text data, see Prepare Text Data for Analysis.

Input words, specified as a string vector, character vector, or cell array of character vectors. If you specify words as a character vector, then the function treats the argument as a single word.

Data Types: string | char | cell

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Example: 'DocumentsIn','rows' specifies the orientation of the output documents as rows.

Orientation of output documents in the frequency count matrix, specified as the comma-separated pair consisting of 'DocumentsIn' and one of the following:

  • 'rows' – Return a matrix of frequency counts with rows corresponding to documents.

  • 'columns' – Return a transposed matrix of frequency counts with columns corresponding to documents.

Data Types: char

Indicator for forcing output to be returned as cell array, specified as the comma separated pair consisting of 'ForceCellOutput' and true or false.

Data Types: logical

Output Arguments

collapse all

Word or n-gram counts, returned as a sparse matrix of nonnegative integers or a cell array of sparse matrices.

If bag is a non-scalar array or 'ForceCellOutput' is true, then the function returns the outputs as a cell array of sparse matrices. Each element in the cell array is matrix of word or n-gram counts of the corresponding element of bag.

Introduced in R2017b