Problem 2043. Six Steps to PCA - Step 1: Centre and Standardize

Created by Neil Caithness

Solve Later

{"relationships":[{"relationshipType":"http://schemas.mathworks.com/matlab/code/2013/relationships/document","targetMode":"","relationshipId":"rId1","target":"/matlab/document.xml"},{"relationshipType":"http://schemas.mathworks.com/matlab/code/2013/relationships/output","targetMode":"","relationshipId":"rId2","target":"/matlab/output.xml"}],"parts":[{"partUri":"/matlab/document.xml","relationship":[],"contentType":"application/vnd.mathworks.matlab.code.document+xml","content":"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<w:document xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\"><w:body><w:p><w:pPr><w:pStyle w:val=\"text\"/></w:pPr><w:r><w:rPr><w:b/></w:rPr><w:t>Introduction</w:t></w:r></w:p><w:p><w:pPr><w:pStyle w:val=\"text\"/></w:pPr><w:r><w:t>Principal Component Analysis (PCA) is a classic among the many methods of multivariate data analysis. Invented in 1901 by Karl Pearson the method is mostly used today as a tool in exploratory data analysis and dimension reduction, but also for making predictive models in machine learning.</w:t></w:r></w:p><w:p><w:pPr><w:pStyle w:val=\"text\"/></w:pPr><w:r><w:rPr><w:b/></w:rPr><w:t>Step 1: Centre and Standardize</w:t></w:r></w:p><w:p><w:pPr><w:pStyle w:val=\"text\"/></w:pPr><w:r><w:t>A first step for many multivariate methods begins by removing the influence of location and scale from variables in the raw data. Also commonly known as the z-scores of</w:t></w:r><w:r><w:t> </w:t></w:r><w:r><w:rPr><w:rFonts w:cs=\"monospace\"/></w:rPr><w:t>X</w:t></w:r><w:r><w:t>,</w:t></w:r><w:r><w:t> </w:t></w:r><w:r><w:rPr><w:rFonts w:cs=\"monospace\"/></w:rPr><w:t>Z</w:t></w:r><w:r><w:t> is a transformation of</w:t></w:r><w:r><w:t> </w:t></w:r><w:r><w:rPr><w:rFonts w:cs=\"monospace\"/></w:rPr><w:t>X</w:t></w:r><w:r><w:t> such that the columns are centered to have mean 0 and scaled to have standard deviation 1 (unless a column of</w:t></w:r><w:r><w:t> </w:t></w:r><w:r><w:rPr><w:rFonts w:cs=\"monospace\"/></w:rPr><w:t>X</w:t></w:r><w:r><w:t> is constant, in which case that column of</w:t></w:r><w:r><w:t> </w:t></w:r><w:r><w:rPr><w:rFonts w:cs=\"monospace\"/></w:rPr><w:t>Z</w:t></w:r><w:r><w:t> is constant at 0). Strictly speaking, z-scores are based on population parameters, whereas the analogous calculation based on sample mean and standard deviation is the Student's t-statistic.</w:t></w:r></w:p><w:p><w:pPr><w:pStyle w:val=\"text\"/></w:pPr><w:r><w:rPr><w:b/></w:rPr><w:t>Task</w:t></w:r></w:p><w:p><w:pPr><w:pStyle w:val=\"text\"/></w:pPr><w:r><w:t>Write a function to centre and standardize the input matrix</w:t></w:r><w:r><w:t> </w:t></w:r><w:r><w:rPr><w:rFonts w:cs=\"monospace\"/></w:rPr><w:t>X</w:t></w:r><w:r><w:t>, returning as the output a structure with the following fields:</w:t></w:r></w:p><w:p><w:pPr><w:pStyle w:val=\"ListParagraph\"/><w:numPr><w:numId w:val=\"1\"/></w:numPr></w:pPr><w:r><w:rPr><w:rFonts w:cs=\"monospace\"/></w:rPr><w:t>Z</w:t></w:r><w:r><w:t>: the centred and standardized matrix corresponding to the input</w:t></w:r><w:r><w:t> </w:t></w:r><w:r><w:rPr><w:rFonts w:cs=\"monospace\"/></w:rPr><w:t>X</w:t></w:r></w:p><w:p><w:pPr><w:pStyle w:val=\"ListParagraph\"/><w:numPr><w:numId w:val=\"1\"/></w:numPr></w:pPr><w:r><w:rPr><w:rFonts w:cs=\"monospace\"/></w:rPr><w:t>Mu</w:t></w:r><w:r><w:t>: a vector of the original means of columns of</w:t></w:r><w:r><w:t> </w:t></w:r><w:r><w:rPr><w:rFonts w:cs=\"monospace\"/></w:rPr><w:t>X</w:t></w:r></w:p><w:p><w:pPr><w:pStyle w:val=\"ListParagraph\"/><w:numPr><w:numId w:val=\"1\"/></w:numPr></w:pPr><w:r><w:rPr><w:rFonts w:cs=\"monospace\"/></w:rPr><w:t>Sigma</w:t></w:r><w:r><w:t>: a vector of the original standard deviations of columns of</w:t></w:r><w:r><w:t> </w:t></w:r><w:r><w:rPr><w:rFonts w:cs=\"monospace\"/></w:rPr><w:t>X</w:t></w:r></w:p><w:p><w:pPr><w:pStyle w:val=\"text\"/></w:pPr><w:r><w:rPr><w:b/></w:rPr><w:t>Tips</w:t></w:r></w:p><w:p><w:pPr><w:pStyle w:val=\"ListParagraph\"/><w:numPr><w:numId w:val=\"1\"/></w:numPr></w:pPr><w:r><w:t>Matlab's</w:t></w:r><w:r><w:t> </w:t></w:r><w:r><w:rPr><w:rFonts w:cs=\"monospace\"/></w:rPr><w:t>zscore</w:t></w:r><w:r><w:t> function is part of the Stats Toolbox which is not available in Cody. You'll have to write your own.</w:t></w:r></w:p><w:p><w:pPr><w:pStyle w:val=\"ListParagraph\"/><w:numPr><w:numId w:val=\"1\"/></w:numPr></w:pPr><w:r><w:t>You should take care to avoid division by zero when a column is invariant.</w:t></w:r></w:p><w:p><w:pPr><w:pStyle w:val=\"text\"/></w:pPr><w:r><w:rPr><w:b/></w:rPr><w:t>Following problems in the series</w:t></w:r></w:p><w:p><w:pPr><w:pStyle w:val=\"ListParagraph\"/><w:numPr><w:numId w:val=\"1\"/></w:numPr></w:pPr><w:hyperlink w:docLocation=\"http://www.mathworks.co.uk/matlabcentral/cody/problems/2049-six-steps-to-pca-step-2-covariance\"><w:r><w:t>Step 2: Covariance</w:t></w:r></w:hyperlink></w:p></w:body></w:document>"},{"partUri":"/matlab/output.xml","contentType":"text/xml","content":"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\" ?><embeddedOutputs><metaData><evaluationState>manual</evaluationState><layoutState>code</layoutState><outputStatus>ready</outputStatus></metaData><outputArray type=\"array\"/><regionArray type=\"array\"/></embeddedOutputs>"}]}

IntroductionPrincipal Component Analysis (PCA) is a classic among the many methods of multivariate data analysis. Invented in 1901 by Karl Pearson the method is mostly used today as a tool in exploratory data analysis and dimension reduction, but also for making predictive models in machine learning.Step 1: Centre and StandardizeA first step for many multivariate methods begins by removing the influence of location and scale from variables in the raw data. Also commonly known as the z-scores of <tt>X</tt>, <tt>Z</tt> is a transformation of <tt>X</tt> such that the columns are centered to have mean 0 and scaled to have standard deviation 1 (unless a column of <tt>X</tt> is constant, in which case that column of <tt>Z</tt> is constant at 0). Strictly speaking, z-scores are based on population parameters, whereas the analogous calculation based on sample mean and standard deviation is the Student's t-statistic.TaskWrite a function to centre and standardize the input matrix <tt>X</tt>, returning as the output a structure with the following fields:<ul><li><tt>Z</tt>: the centred and standardized matrix corresponding to the input <tt>X</tt></li><li><tt>Mu</tt>: a vector of the original means of columns of <tt>X</tt></li><li><tt>Sigma</tt>: a vector of the original standard deviations of columns of <tt>X</tt></li></ul>Tips<ul><li>Matlab's <tt>zscore</tt> function is part of the Stats Toolbox which is not available in Cody. You'll have to write your own.</li><li>You should take care to avoid division by zero when a column is invariant.</li></ul>Following problems in the series<ul><li><a href = "http://www.mathworks.co.uk/matlabcentral/cody/problems/2049-six-steps-to-pca-step-2-covariance">Step 2: Covariance</a></li></ul>

Solve

Solution Stats

244 Solutions
35 Solvers

Last Solution submitted on Jul 25, 2020

Last 200 Solutions

Problem Comments

1 Comment

1 Comment

Jean-Marie Sainthillier on 10 Feb 2014

Your definition of a constant (or invariant) data with rand is problematic. If you increase the size of your data (n=1000, n=10000...), you can always increase the deviations (so what threshold for sigma ?). I think that with real data, this artifact isn't possible. No ?

Solution Comments

Solution 370564

1 Comment

1 Comment

Neil Caithness on 17 Dec 2013

Nice solution that looks pretty nifty but it actually produces an unsafe result, just like Matlab's own zscore. It passed on it's first run trough the test suite but fails on some number of repeats. I've now added a second pass with a know random seed.

Problem Recent Solvers35

Problem Tags

Problem 2043. Six Steps to PCA - Step 1: Centre and Standardize

Solution Stats

Last 200 Solutions

Problem Comments

Solution Comments

Problem Recent Solvers35

Suggested Problems

More from this Author1

Problem Tags

Problem 2043. Six Steps to PCA - Step 1: Centre and Standardize

Solution Stats

Last 200 Solutions

Problem Comments

Solution Comments

Problem Recent Solvers35

Suggested Problems

More from this Author1

Problem Tags

Players