Creating New Kinds of System Objects for File Input and Output

This example shows how to create and use two different System objects to facilitate the streaming of data in and out of MATLAB: dspdemo.TextFileReader and dspdemo.TextFileWriter.

The objects discussed in this example address a number of realistic use cases, and they can be customized to achieve advanced and more specialized tasks.


This example shows how to create and use new types of System objects for file reading and writing. Internally, these System objects use standard low-level file I/O functions available in MATLAB (e.g. fscanf, fprintf, fread, fwrite). By abstracting away most usage details of those functions, they aim to make the task of reading and writing streamed data simpler and more efficient.

This example includes the use of a number of advanced constructs to author System objects. For a more basic introduction to authoring System objects, please refer to the examples in the section Define New System Objects, of the DSP System Toolbox documentation.

The System Object™ Interface

System objects are MATLAB classes that derive from matlab.System. As a result, System objects all inherit a common public interface, which includes the following standard methods:

  • setup - to initialize the object, typically at the beginning of a simulation

  • reset - to clear the internal state of the object, bringing it back to its default post-initialization status

  • step - to execute the core functionality of the object, optionally accepting some input and/or returning some output

  • release - to release any resources (e.g. memory, hardware, or OS-specific) used internally by the object

When you create new kinds of System objects, you provide specific implementations for all the preceding methods to determine its behavior.

In this example we discuss the internal structure and the use of the following two System objects:

Definition of the Class dspdemo.TextFileReader

All System objects in the previous list share a common structure. For example dspdemo.TextFileReader includes the following sections

1. A class definition statement, which implies this class is derived from both matlab.System and matlab.system.mixin.FiniteSource.

classdef TextFileReader < matlab.System & matlab.system.mixin.FiniteSource
  • matlab.System is required, and is the base class for all System objects

  • matlab.system.mixin.FiniteSource indicates this class is a signal source with a finite number of data samples. This implies that, in addition to the usual interface, the System object will also expose the method isDone. When isDone returns true the object reached the end of the available data.

2. A number of public properties. In this case two are nontunable (they cannot be changed after the first call to step) and all have a default value. Default values are assigned to the corresponding properties when nothing else is specified by the user. Public properties can be changed by the user to adjust the behaviour of the object to his or her particular application.

   properties (Nontunable)
       Filename   = 'tempfile.txt'
       HeaderLines = 4
       DataFormat = '%g'
       Delimiter = ','
       SamplesPerFrame = 1024
       PlayCount = 1

3. A number of private properties. These are not visible to the user, and can serve a number of purposes, including

  • To keep hold of values computed only occasionally (e.g. at initialization time, when setup is called or when step is called for the first time) and then consumed by subsequent calls to step. This can save re-computing them at runtime, and hence improve the performance of the core functionality

  • To define the internal state of the object. For example pNumEofReached stores the number of times that the end-of-file indicator was reached

   properties(Access = private)
       pFID = -1
       pNumEofReached = 0

4. A constructor. This is called when a new instance of dspdemo.TextDataReader is created by the user. Calling the method setProperties within the constructor allows users to set the properties of the object by providing name-value pairs at construction.

       function obj = TextFileReader(varargin)
           setProperties(obj, nargin, varargin{:});

5. A number of overridden methods from the matlab.System base class. The public methods common to all System objects each have corresponding protected methods that they call internally. The names of these protected methods all include an Impl postfix. They can be implemented when defining the class to program the specific behaviour of the particular System object.

For more information on the correspondence between the standard public methods and their internal implementations, please refer to Methods Timing.

For example, the particular implementation methods that are overridden for dspdemo.TextFileReader are

  • setupImpl

  • resetImpl

  • stepImpl

  • releaseImpl

  • isDoneImpl

  • processTunedPropertiesImpl

  • loadObjectImpl

  • saveObjectImpl

6. A number of private methods. These methods are only accessible from within other methods of the same class. They can be used to make the rest of the code more readable. They can also improve code reusability, by grouping under separate routines code that is used multiple times in different parts of the class.

Write and Read Data - Introduction to the Example

The code that follows gives a simple demonstration of how these new objects could be used. The following tasks are shown

  • Create a text file containing the samples of two different sinusoidal signals using dspdemo.TextFileWriter

  • Read from the text file using dspdemo.TextFileReader and write to a second file in binary form, this time using dsp.BinaryFileWriter

  • Read the signal samples cyclically from the new binary file using dsp.BinaryFileReader, and analyze the results graphically.

Create a Simple Text File Containing the Desired Data

To start, a new text file is created to store two sinusoidal signals with frequencies 50 Hz and 60 Hz, respectively. For each signal, the data stored will be composed of 800 samples at a sampling rate of 8 kHz.

The following prepares the data

% Create data samples
fs = 8000;
tmax = 0.1;
t = (0:1/fs:tmax-1/fs)';
N = length(t);
f = [50,60];
data = sin(2*pi*t*f);

% Optionally, form a header string to describe the data in a readable way
% for future use
fileheader = sprintf(['The following contains %d samples of two ',...
    'sinusoids,\nwith frequencies %d Hz and %d Hz and a sample rate of',...
    ' %d kHz\n\n'], N, f(1),f(2),fs/1000);

To store the signal to a text file, create an instance of a text file writer. The constructor of dspdemo.TextFileWriter needs the name of the target file and some optional parameters, which can be passed in as name-value pairs.

TxtWriter = dspdemo.TextFileWriter('Filename','sinewaves.txt',...
    'Header',fileheader) %#ok<NOPTS>
TxtWriter = 

  dspdemo.TextFileWriter with properties:

      Filename: 'sinewaves.txt'
        Header: 'The following contains 800 samples of two sinusoids,...'
    DataFormat: '%.18g'
     Delimiter: ','

dspdemo.TextFileWriter writes data to delimiter-separated ASCII files. Its public properties include the following

  • Filename: the name of the file to be written. If a file with this name already exists, it is overwritten. When operations start, the object begins writing to the file immediately following the header - it then appends new data at each subsequent call to step, until it is released. Calling reset resumes writing from the beginning of the file.

  • Header: a character string, often composed of multiple lines and terminated by a newline character (\n). This is specified by the user and can be modified to embed human-readable information that describes the actual data.

  • DataFormat: the format used to store each data sample. This can take any value assignable as Conversion Specifier within the formatSpec string used by the built-in MATLAB function fprintf. DataFormat applies to all channels written to the file. The default value for this property is '%.18g', which allows saving double precision floating point data in full precision.

  • Delimiter: the character used to separate samples from different channels at the same time instant. Every line of the written file maps to a time instant, and it includes as many samples as the number of channels provided as input (i.e. the number of columns in the matrix input passed to step).

To write all the available data to the file, a single call to step can be used as follows

% Write to file with data as input
% Note: |TxtWriter(data)| is equivalent to |step(TxtWriter, data)|

% Release control of file

The data is now stored in the new file. To inspect the file visually type edit('sinewaves.txt'). Because of the header, note that the data starts on line 4, following the 3 lines of the header.

In this simple case, the length of the whole signal is small and it fits comfortably on system memory. Therefore the data can be created all at once and written to a file in a single step as just shown.

There are cases when this approach is not possible or practical. For example the data may be too large to fit into a single MATLAB variable (i.e. too large to fit on system memory), or it may be created cyclically in a loop, or streamed into MATLAB from an external source. In all these cases it can be convenient to stream the data into the file by using an approach similar to the following

% Use a streamed sine wave generator to create a frame of data per step
frameLength = 32;
SineWave = dsp.SineWave('Frequency',[50,60], 'SampleRate', fs, ...
    'SamplesPerFrame', frameLength);

% Run the desired number of iterations to create the data and store it into
% the file
tmax = 10; % Write more data in this scenario
t = (0:1/fs:tmax-1/fs)';
N = length(t);
data = sin(2*pi*t*f);
numCycles = N/frameLength;
for k = 1:numCycles
    dataFrame = SineWave();

% Release control of file and sine wave generator

Read from Existing Text File and Write to New Binary File

The next step consists in reading the data from the newly-created file, and writing it into a new binary file.

To read from the text file, create an instance of dspdemo.TextFileReader.

% Create a text file reader
TxtReader = dspdemo.TextFileReader('Filename','sinewaves.txt',...
    'HeaderLines',3,'SamplesPerFrame',frameLength) %#ok<NOPTS>
TxtReader = 

  dspdemo.TextFileReader with properties:

           Filename: 'sinewaves.txt'
        HeaderLines: 3
         DataFormat: '%g'
          Delimiter: ','
    SamplesPerFrame: 32
          PlayCount: 1

dspdemo.TextFileReader reads numeric data from delimiter-separated ASCII files. Its properties are similar to those of dspdemo.TextFileWriter. Some differences follow

  • HeaderLines captures the number of lines used by the header within the file specified in Filename. The first call to step starts reading from line number HeaderLines+1. Subsequent calls to step keep reading from the line immediately following the previously read line. Calling reset will resume reading from line HeaderLines+1.

  • Delimiter is again the character used to separate samples from different channels at the same time instant. In this case it is also used to determine the number of data channels stored in the file: when step is first called, the object counts the number of Delimiter characters at line HeaderLines+1, say numDel; it then assumes for every time instant it needs to read numChan = numDel+1 numeric values with format DataFormat. The matrix returned by step has size SamplesPerFrame x numChan.

  • SamplesPerFrame is the number of lines read by each call to step, i.e. the number of rows of the matrix returned as output. When the last available data rows are reached, it can happen that they are fewer than the required SamplesPerFrame. In that case, the available data are padded with zeros to obtain a matrix of size SamplesPerFrame x numChan. Once all the data are read, step simply returns zeros(SamplesPerFrame,numChan) until reset or release is called.

  • PlayCount is the number of times the data in the file is read cyclically. If the object reaches the end of the file, and the file has not yet been read a number of times equal to PlayCount, reading resumes from the beginning of the data (i.e. line HeaderLines+1). If the last lines of the file do not provide enough samples to form a complete output matrix of size SamplesPerFrame x numChan, then the frame is completed using the initial data. Once the file is read PlayCount times, the output matrix returned by step is filled with zeros, and all calls to isDone return true unless reset or release is called. To loop through the available data indefinitely, PlayCount can be set to Inf.

To write to a new binary file, create an instance of dsp.BinaryFileWriter

BinWriter = dsp.BinaryFileWriter('Filename','sinewaves.bin',...

The data passed as input to dsp.BinaryFileWriter is stored using its own data type. For example a double-precision floating-point input will use 8 bytes per sample. Data samples are stored in time order and multiple channels are interleaved. As a consequence, the input matrices passed to step are appended to the file using a row-major approach (i.e. the first row is stored first, left to right, then the second row, and so on).

To transfer the data from the text file into the binary file, the more general streamed approach is used. This is also relevant to dealing with very large data files.

% Write binary data using single-precision floating point
% Preallocate a data frame with |frameLength| rows and 2 columns
dataFrame = zeros(frameLength,2,'single');

% Read from the text file and write to the binary file, whilst data is
% present in the source text file. Notice how the method |isDone| is used
% to control the execution of the while loop
    dataFrame(:) = TxtReader();

% Release control of both files

Read Content of Binary File Cyclically

sinewaves.bin holds a whole number of periods for both sine waves, i.e. 500 periods at 50 Hz and 600 at 60 Hz. Such signals can be read cyclically and used to generate sine waves of arbitrary length. In the last part of this demonstration dsp.BinaryFileReader is used to do exactly this. As data are read, the two sine waves are visualized in the time domain and their product is analyzed in the frequency domain.

frameLength = 1024;

% Create an instance of a binary file reader
BinReader = dsp.BinaryFileReader('Filename','sinewaves.bin',...
    'SamplesPerFrame',frameLength) %#ok<NOPTS>
header = BinReader.readHeader.Info;
% Display the header read from the file:
BinReader = 

  dsp.BinaryFileReader with properties:

           Filename: 'sinewaves.bin'
    HeaderStructure: [1x1 struct]
    SamplesPerFrame: 1024
        NumChannels: 2
           DataType: 'single'
      IsDataComplex: false

ans =

    'The following contains 800 samples of two sinusoids,
     with frequencies 50 Hz and 60 Hz and a sample rate of 8 kHz

The interface of dsp.BinaryFileReader is mostly self-explanatory. The following are worth noticing

  • HeaderStructure defines the expected header at the beginning of the file.

  • NumChannels specifies how many interleaved samples are expected for each time instant. It also determines the number of columns of the output matrix returned by the object. When executed, the object returns a matrix of size SamplesPerFrame x NumChannels, stored in the file in a row-major fashion.

The following creates components that help with the visual analysis of the streamed signals, both in the time and in the frequency domain

For time-domain visualization, an instance of dsp.TimeScope is created. This is used to plot all data frames for both sine waves, as they are read from the file.

TimeScope = dsp.TimeScope('SampleRate',fs,'TimeSpan',frameLength/fs,...
    'ShowGrid',true,'YLimits',[-1 1],'TimeSpanOverrunAction','Scroll');

For frequency domain visualization, an instance of dsp.SpectrumAnalyzer is created. This is used to analyze the spectrum of the product between the two sine waves, expecting two tonal components at (60-50) Hz and (60+50) Hz, respectively.

Because the ratio between the frequencies in the signal and the sample rate is very low, the signals are first decimated by a factor of 16 with an instance of dsp.SampleRateConverter. This brings the sample rate down to 500 Hz and makes the frequency components of the target signal more easily identifiable using a standard spectral analysis.

RateConverter = dsp.SampleRateConverter('InputSampleRate', 8000, ...
    'OutputSampleRate', 500, 'Bandwidth', 100);

% Note how a single spectral snapshot with the following settings
% requires 16384 input samples, far more than the 800 actually stored in
% the file
SpectAnalyzer = dsp.SpectrumAnalyzer(...
    'FrequencyResolutionMethod', 'WindowLength', 'WindowLength', 1024,...
    'FFTLengthSource', 'Property', 'FFTLength', 2048, 'SampleRate',500,...
    'PlotAsTwoSidedSpectrum', false, 'SpectralAverages', 16);

The following while loop reads the data from the file and visualizes the signals as they are read. It runs for 10 minutes worth of simulation time, regardless of the actual number of samples in the file. The plots in the two streamed visualizations match the expected behaviour.

simtime = 0;
% Run the loop until the simulation time is less than 10*60 seconds
while(simtime < 600)
    % Rewind file if there are no more new samples
    if isDone(BinReader)
    % Read from binary file, 1024 samples per frame
    dataFrame = BinReader();
    % Visualize a single frame of both channels in the time domain
    % Decimate the two channels down to a new sample rate of 500 Hz,
    % resulting in a new frame length of 64 samples
    dataDecimated = RateConverter(dataFrame);
    % Analyze the product of the two sine waves in the frequency domain, by
    % accumulating multiple data frames internally and updating the
    % visualization when ready

    % Update value of simulation time elapsed
    simtime = simtime + frameLength/fs;

% Release control of files and scopes


This example illustrated how to author and use System objects to read from and write to numeric data files. All objects used (i.e. dspdemo.TextFileReader, dspdemo.TextFileWriter, dsp.BinaryFileReader, and dsp.BinaryFileWriter) can be edited to perform special-purpose file reading and writing operations.

For more information on authoring System objects for custom algorithms, see Create System Objects.