Main Content

Preprocessing Data

Data cleaning, smoothing, grouping

Data can require preprocessing techniques to ensure accurate, efficient, or meaningful analysis. Data cleaning refers to methods for finding, removing, and replacing bad or missing data. Detecting local extrema and abrupt changes can help to identify significant data trends. Smoothing and detrending are processes for removing noise and polynomial trends from data, while scaling changes the bounds of the data. Grouping and binning methods identify data characteristics by groups.

Apps

Data CleanerPreprocess and organize column-oriented data (Since R2022a)

Live Editor Tasks

Clean Missing DataFind, fill, or remove missing data in the Live Editor (Since R2019b)
Clean Outlier DataFind, fill, or remove outliers in the Live Editor (Since R2019b)
Find Change PointsFind abrupt changes in data in the Live Editor (Since R2019b)
Find Local ExtremaFind local maxima and minima in the Live Editor (Since R2019b)
Smooth DataSmooth noisy data in the Live Editor (Since R2019b)
Find and Remove TrendsFind and remove polynomial or periodic trends from data in the Live Editor (Since R2019b)
Normalize DataCenter and scale data in the Live Editor (Since R2021b)
Pivot TableSummarize tabular data in pivoted table in the Live Editor (Since R2023b)
Compute by GroupSummarize, transform, or filter by group in the Live Editor (Since R2021b)

Functions

expand all

anymissingDetermine if any array element is missing (Since R2022a)
ismissingFind missing values
rmmissingRemove missing entries
fillmissingFill missing entries
fillmissing2Fill missing entries in 2-D data (Since R2023a)
missingCreate missing values
standardizeMissingInsert standard missing values
isoutlierFind outliers in data
filloutliersDetect and replace outliers in data
rmoutliersDetect and remove outliers in data
movmadMoving median absolute deviation
ischangeFind abrupt changes in data
islocalminFind local minima
islocalmaxFind local maxima
smoothdataSmooth noisy data
smoothdata2 Smooth noisy data in two dimensions (Since R2023b)
movmeanMoving mean
movmedianMoving median
detrendRemove polynomial trend
trenddecompFind trends in data (Since R2021b)
normalizeNormalize data
rescaleScale range of array elements
discretizeGroup data into bins or categories
pivotSummarize tabular data in pivoted table (Since R2023a)
groupsummaryGroup summary computations
groupcountsNumber of group elements (Since R2019a)
groupfilterFilter by group (Since R2019b)
grouptransformTransform by group
histcountsHistogram bin counts
histcounts2Bivariate histogram bin counts
findgroupsFind groups and return group numbers
splitapplySplit data into groups and apply function
rowfunApply function to table or timetable rows
varfunApply function to table or timetable variables
accumarrayAccumulate vector elements

Topics