Find K-Means average distance from points within a cluster to their respective centroid?

I've run the k-means algorithm on my data and it returns "D" = distances from each point in the data set to every centroid, and "sumd" = within-cluster sums of point-to-centroid distances. However, I would like to find a way to calculate the mean distance from points within a cluster to their centroids. What is the best way to do this?

Answer 1

[Image Analyst]

The second argument of kmeans is the cluster centroids. It's an easy matter to get the distance from each point in a class to the centroid. Let's say you have 3 classes and data has two coordinates, x, and y.

numClasses = 3;
[classIndexes, clusterCentroids] = kmeans(data, numClasses);
meanDistances = zeros(numClasses, 1);
for k = 1 : numClasses
    % Get x and y of centroid.
    xc = clusterCentroids(k, 1);
    yc = clusterCentroids(k, 2);
    % Get the x and y coordinates of points within this class only.
    inClass = classIndexes == k; % Indexes of points that were assigned to this class.
    x = data(inClass, 1); % Get not ALL x, but just x within this one class.
    y = data(inClass, 2); % Get not ALL y, but just y within this one class.
    % Get distances of all points in the class to the centroid for this class.
    distances = sqrt((x - xc) .^ 2 + (y - yc) .^ 2);
    meanDistances(k) = mean(distances);
end

The concept can be generalized to any number of features. Instead of 2 you can have 3 or 4 or 20. It doesn't matter - just make the obvious changes to generalize it. How many features do you have?

[D F]

Thanks so much! I have 3 classes with x, y, and z coordinates. To clarify, in your answer when you say "for k=1 : numClasses" does k=1 denote the first cluster out of 3?

[Image Analyst]

Yep, fine. So I assume you made these changes:

numClasses = 3;
[classIndexes, clusterCentroids] = kmeans(data, numClasses);
meanDistances = zeros(numClasses, 1);
for k = 1 : numClasses
    % Get x, y, and z of centroid.
    xc = clusterCentroids(k, 1);
    yc = clusterCentroids(k, 2);
    zc = clusterCentroids(k, 3);
    % Get the x, y, and z coordinates of points within this one class only.
    inClass = classIndexes == k; % Indexes of points that were assigned to this class.
    x = data(inClass, 1); % Get not ALL x, but just x within this one class.
    y = data(inClass, 2); % Get not ALL y, but just y within this one class.
    z = data(inClass, 3); % Get not ALL z, but just z within this one class.
    % Get distances of all points in the class to the centroid for this class.
    distances = sqrt((x - xc) .^ 2 + (y - yc) .^ 2 + (z - zc) .^ 2);
    meanDistances(k) = mean(distances);
end

[Image Analyst]

Yes, k goes from 1 to 2 to 3, which processes each cluster in turn.

numPoints = 15;
c1 = 2 + randn(numPoints, 3);
c2 = 6 + randn(numPoints, 3);
c3 = 10 + randn(numPoints, 3);
data = [c1;c2;c3];
x = data(:, 1);
y = data(:, 2);
z = data(:, 3);
scatter3(x, y, z, 10, 'filled')
xlabel('x');
ylabel('y');
zlabel('z');
numClasses = 3;
[classIndexes, clusterCentroids] = kmeans(data, numClasses);
meanDistances = zeros(numClasses, 1);
for k = 1 : numClasses
    fprintf('Getting centroid-to-point distances of points in cluster #%d.\n', k);
    % Get x, y, and z of centroid.
    xc = clusterCentroids(k, 1);
    yc = clusterCentroids(k, 2);
    zc = clusterCentroids(k, 3);
    % Get the x, y, and z coordinates of points within this one class only.
    inClass = classIndexes == k; % Indexes of points that were assigned to this class.
    x = data(inClass, 1); % Get not ALL x, but just x within this one class.
    y = data(inClass, 2); % Get not ALL y, but just y within this one class.
    z = data(inClass, 3); % Get not ALL z, but just z within this one class.
    % Get distances of all points in the class to the centroid for this class.
    distances = sqrt((x - xc) .^ 2 + (y - yc) .^ 2 + (z - zc) .^ 2);
    meanDistances(k) = mean(distances);
end
% Optional plot lines from each point to its centroid.
cla
hold on;
colors = {'r', 'g', 'b'};
for k = 1 : numClasses
    fprintf('Plotting points of cluster #%d.\n', k);
    % Get x, y, and z of centroid.
    xc = clusterCentroids(k, 1);
    yc = clusterCentroids(k, 2);
    zc = clusterCentroids(k, 3);
    % Get the x, y, and z coordinates of points within this one class only.
    inClass = classIndexes == k; % Indexes of points that were assigned to this class.
    x = data(inClass, 1); % Get not ALL x, but just x within this one class.
    y = data(inClass, 2); % Get not ALL y, but just y within this one class.
    z = data(inClass, 3); % Get not ALL z, but just z within this one class.
    % Get distances of all points in the class to the centroid for this class.
    for k2 = 1 : length(x)
        plot3([xc, x(k2)], [yc, y(k2)], [zc, z(k2)], '.-', 'Color', colors{k}, 'MarkerSize', 18)
    end
end
hold off;
caption = sprintf('%d Clusters with Centroids', numClasses);
title(caption)

[D F]

Thanks so much for the detailed answer!

[D F]

When changing the value of k to 2 or 3 in

for k = 1 : numClasses

I noticed that the values returned for xc/yc/zc and x/y/z remain the same and don't change for the new centroid and points. Do you know how to solve this? I greatly appreciate the help!

[Image Analyst]

Please post your data. How many classes does it reasonably have? If k=numClasses was only 2, then you should definitely have 2 rows in the centroids matrix instead of 3 because you're telling it that there must be only 2 clusters. I don't see any way it is unchanged.

[D F]

It has 3 classes and the centroids matrix has 3 rows, but the matrix of the cluster points and the point-to-centroid distances appears to only be for the first cluster, even after changing k = __ : numClasses to find the points/distances for the second and third cluster. I also had it replicate to match the centroids I had found previously.

[classIndexes, clusterCentroids] = kmeans(V1_1, numClasses,'replicates', 100);

Here is my data:

51 -46 56

-48 -43 44

-54 20 -4

-39 29 47

57 -40 11

-45 -34 38

-48 -61 29

9 56 2

-18 41 32

-6 -49 47

-3 26 26

-30 -7 47

51 -34 32

-51 -64 8

27 -40 47

-9 38 2

-12 -64 62

-39 -67 -43

39 -79 41

54 -40 59

66 -28 -13

-63 -49 38

39 56 2

48 -64 -37

-30 50 14

-3 -49 38

-54 -52 17

51 38 8

-21 -64 -55

63 -16 41

0 -55 32

9 62 2

39 -7 41

-69 -25 -22

3 20 26

-54 -1 11

-9 -70 62

-6 53 -7

30 -79 44

-66 -40 29

69 -31 26

-57 8 -13

-24 47 5

54 2 5

27 -91 32

-24 5 -13

66 -37 14

Centroids:

-36.3529411764706 -46.7058823529412 22.4705882352941

-10.2000000000000 36.6000000000000 8.60000000000000

49.4000000000000 -42 26.6000000000000

[Image Analyst]

I'm not sure what you mean. It seems to work exactly as I expected.

% Demo by Image Analyst, December, 2021.
clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clearvars;
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 16;
fprintf('Beginning to run %s.m ...\n', mfilename);

data = [...
51	-46	56
-48	-43	44
-54	20	-4
-39	29	47
57	-40	11
-45	-34	38
-48	-61	29
9	56	2
-18	41	32
-6	-49	47
-3	26	26
-30	-7	47
51	-34	32
-51	-64	8
27	-40	47
-9	38	2
-12	-64	62
-39	-67	-43
39	-79	41
54	-40	59
66	-28	-13
-63	-49	38
39	56	2
48	-64	-37
-30	50	14
-3	-49	38
-54	-52	17
51	38	8
-21	-64	-55
63	-16	41
0	-55	32
9	62	2
39	-7	41
-69	-25	-22
3	20	26
-54	-1	11
-9	-70	62
-6	53	-7
30	-79	44
-66	-40	29
69	-31	26
-57	8	-13
-24	47	5
54	2	5
27	-91	32
-24	5	-13
66	-37	14]
x = data(:, 1);
y = data(:, 2);
z = data(:, 3);
subplot(2, 3, 1);
scatter3(x, y, z, 10, 'filled')
xlabel('x');
ylabel('y');
zlabel('z');
title('Original Data')
grid on;

numClasses = 3;
[classIndexes, clusterCentroids] = kmeans(data, numClasses,'replicates', 100);
meanDistances = zeros(numClasses, 1);
for classIndex = 1 : numClasses
    fprintf('Getting centroid-to-point distances of points in cluster #%d.\n', classIndex);
    % Get x, y, and z of centroid.
    xc = clusterCentroids(classIndex, 1);
    yc = clusterCentroids(classIndex, 2);
    zc = clusterCentroids(classIndex, 3);
    % Get the x, y, and z coordinates of points within this one class only.
    inClass = classIndexes == classIndex; % Indexes of points that were assigned to this class.
    x = data(inClass, 1); % Get not ALL x, but just x within this one class.
    y = data(inClass, 2); % Get not ALL y, but just y within this one class.
    z = data(inClass, 3); % Get not ALL z, but just z within this one class.
    % Get distances of all points in the class to the centroid for this class.
    distances = sqrt((x - xc) .^ 2 + (y - yc) .^ 2 + (z - zc) .^ 2);
	subplot(2, 3, classIndex + 3);
	bar(distances);
	xlabel('Point Number');
	ylabel('Distance from Centroid');
	caption = sprintf('Distances for class #%d', classIndex);
	grid on;
	title(caption)
    meanDistances(classIndex) = mean(distances);
end
% Optional plot lines from each point to its centroid.
subplot(2, 3, 2:3);
colors = {'r', 'g', 'b'};
for classIndex = 1 : numClasses
    fprintf('Plotting points of cluster #%d.\n', classIndex);
    % Get x, y, and z of centroid.
    xc = clusterCentroids(classIndex, 1);
    yc = clusterCentroids(classIndex, 2);
    zc = clusterCentroids(classIndex, 3);
    % Get the x, y, and z coordinates of points within this one class only.
    inClass = classIndexes == classIndex; % Indexes of points that were assigned to this class.
    x = data(inClass, 1); % Get not ALL x, but just x within this one class.
    y = data(inClass, 2); % Get not ALL y, but just y within this one class.
    z = data(inClass, 3); % Get not ALL z, but just z within this one class.
    % Get distances of all points in the class to the centroid for this class.
    for k2 = 1 : length(x)
        plot3([xc, x(k2)], [yc, y(k2)], [zc, z(k2)], '.-', 'Color', colors{classIndex}, 'MarkerSize', 18)
		hold on;
    end
end
grid on;
xlabel('x');
ylabel('y');
zlabel('z');
hold off;
caption = sprintf('%d Clusters with Centroids', numClasses);
title(caption)

Look, I've had it show the point-to-centroid distances for each class (in bar charts in the bottom row) and you can see they're different. The distances are not all the same as for the first cluster. You can see in the loop where I compute distances that xc, yc, zc, and x, y, and z DO change with each iteration (class). So I'm not sure why you're saying that.

[D F]

Oh okay, I see now, thank you. Also I was just curious based on your username if you have any experience with fMRI data. I have the locations of the 3 cluster centroids found from k-means, the MNI coordinates that were used to calculate the centroids, and the resultant p values and cluster size in voxels for each of the 3 clusters. I was wondering if you knew of a program/toolbox that would enable me to visualize these clusters of varying sizes on a "standard" brain? I don't have access to the imaging data/files currently. Thank you.

[Image Analyst]

I've worked on MRI images before but not specifically fMRI that I remember.